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ABSTRACT 

The concept of the personnel assessment center is intro- 
duced, explained and evaluated by means of personal interviews 
and a literature review- Next, a proposed design for a proto- 
type assessment center for the Naval Postgraduate School (NPS) 
is offered along with the ancillary design of validation/evalu- 
ation/feedback systems. Finally, a justification, listing of 
tasks to be performed, and anticipated results are provided. 

The reader is introduced to the concept through defini- 
tions, a brief history, and a comprehensive overview of the 
field. Explanation and evaluation of assessment centers is 
accomplished by the reporting of interviews conducted by the 
writer, and the literature review. The assessment center 
design proposal for the NPS reflects the opinions of the 
writer, based on the research conducted during the course of 
this project. 

Short and long term anticipated results are listed which 
cite potential benefits to individual participants, the NPS 
and the Navy. 
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I. STATEMENT OF THE PROBLEM 



During the past fifty years there have been dramatic 
advances in military "hardware" but relatively modest changes 
in personnel management techniques. Although the need to keep 
pace in weapons technology is not disputed, this writer believes 
that the potential return on investment available from improved 
personnel management has escalated sharply in the past few 
years- -to the point where personnel research can now compete 
more successfully with "hard" research for scarce defense 
resources. This belief is based on three factors: (1) The 

contention that people vice hardware are usually the decisive 
factor in combat; (2) The impact of the All Volunteer Force 
concept on the U. S. military; and (3) The implications of 
the Operational, Technical, Managerial System (OTMS) for the 
U. S. Navy. 

The first factor is mentioned (in a business vice military 
context) in Thompson's article [62] where he cites the view 
of D. C. Burnham, Chairman of Westinghouse Electric Corporation 
as follows: "Nearly everyone has the materials, the facilities, 

the techniques available to them, but how they use these things 
depends on their people." Admiral Thomas H. Moorer described 
the issue in a military context in an address to the graduating 
class at the Naval Postgraduate School on March 29, 1974. He 
reported that analyses of major naval battles of World War II 
disclosed that a majority of them were won by the side with 
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technically, and/or numerically, inferior forces. His con- 
clusion was that the personnel involved had made the crucial 
difference . 

The second factor evolves from recent changes in the fabric 
of U. S. society which have brought increasing pressure to 
bear on the military services toward increased consideration 
of individuals. This is compounded by the reality of the All- 
Volunteer concept which has forced the military services into 
head-to-head competition with industry over the nation's human 
resources. The "sanctuaries" (draft, etc.), so long enjoyed 
by the military, have been abolished, and this has thrust the 
military manpower problem onto center stage. 

The third factor results, in part at least, from the other 
two which have begun to force changes in the traditional Naval 
Officer career patterns. The j ack-of- all - trades , master-of- 
none Naval Officer generalist of yesteryear is finding it 
increasingly difficult to cope with the complex and varied 
problems of today’s Navy. Bowling [6] expressed it thus: 

It has been a long-accepted fact that the Navy needs 
both good operators and good managers. What has not been 
accepted or fully recognized is that, in the current tech- 
nological, strategic, and tactical environment- -given the 
complexities of nuclear ships and weapons systems, coupled 
with the relatively short span of an officer's career- -it 
is no longer possible to develop both within the same 
individual . 

He further recommended that the CNO : 

. . . establish the policy that all unrestricted line 

officers specialize in either Operations or Management, 
with the understanding that both specialties will provide 
equal but parallel paths to selection for flag rank in 
the unrestricted line. 
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Although Bowling's arguments have merit, they also have 

drawbacks, and his concept of sea-going operators and shore- 

based managers did not come to pass. Instead, the Operational, 

Technical, Managerial System (OTMS) was established in the 

summer of 1972. The concept, and the rationale behind its 

development were described, in part, as follows [107]: 

. . . Although there are several approaches which could be 
taken to provide specialization across the full spectrum 
of Navy endeavor. Admiral Zumwalt has firmly rejected the 
idea of going to a wet/dry Navy. (Writer's note: This 

was Bowling's proposal.) Navies of other nations have 
tried that approach and found it unsatisfactory. The 
Operational Technical Managerial System has as its goal 
the development of the broad sea and operationally-oriented 
unrestricted line officer blended with the technical exper- 
tise normally found in the restricted line and staff corps 
. . . Before a naval officer should consider becoming a 

career subspecialist (concentration within a subspecialty 
area) , the officer should complete at least two operational 
tours . . . Through a system of evolution we are imple- 
menting . . . development and identification systems to 

meet the wide spectrum of Navy technical and managerial 
requirements . 

These factors illustrate the continuing and increasing 
need for selection (hiring, firing, promoting) , development 
and placement within the U. S. Navy. In this macro context 
the need is for an integrated systems approach to personnel 
management involving many complementary subsystems. This 
study focused on one (proposed) subsystem: the personnel 

assessment center. The approach used was: First, to research 

the philosophies, structures and applications of the assess- 
ment center concept, and evaluate the concept in the are n s of 
selection, development and placement; and, next, to put the 
assessment center in perspective within an "ideal" personnel 
management system, explaining the interfaces and relationships 
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between the assessment center and other subsystems; and, 
finally, to develop a research proposal for the design of a 
prototype personnel assessment center for the management cur- 
riculum of the Naval Postgraduate School. 

Hardesty and Jones [31] offer their opinion as to the 
potential value of the assessment center: 

Because the concern of the assessment center is with 
the organization's leaders of the future, the ultimate 
validity of this program . . . may well be measurable in 

the future success of the companies as contrasted to sim- 
ilar companies who do not make such an effort. The stakes 
are high, but on the other hand they always have been . . . 

If the stakes are considered to be high in the business 
world, can they be otherwise in the military? The professional 
military man is acutely aware of the consequences of finishing 
second in combat. There will always be a need for good men 
and women- -the problems are finding, developing and retaining 
them. This paper is devoted to the examination of one proposed 
solution to those problems: the personnel assessment center. 
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II. BACKGROUND 



A. DEFINITIONS 

A typical industrial assessment center is described by 
Byham [11] as follows: 

In these centers, specially trained managers (and occasion- 
ally psychologists) act as "assessors" who evaluate candi- 
dates for promotion- -either into management or within 
management -- on their potential and their areas of weakness. 
Groups of men pass through series of standardized exercises 
such as management games, in-basket tests, and leaderless 
discussion sessions, while assessors observe their behavior 
closely. 

The assessors discuss each candidate's performance 
separately and then generate a comprehensive report on each 
candidate which management can combine with current per- 
formance information as it sees fit. As well as identifying 
the men most likely to succeed, the assessment reports spell 
out the individual deficiencies of each candidate and suggest 
guidelines for management to use in developing him. 

The U. S. Army Infantry School (USAIS) Assessment Center 
[127] uses this definition: 

The term assessment center is used to describe a process by 
which candidates for a position are put through a series of 
standardized exercises designed to simulate the conditions 
of that position and to show if they have the skills and 
abilities necessary to perform it. 

The assessment process is conducted using a series of 
situational exercises designed to measure leadership behav- 
iors and skills one level higher than the individual has 
performed. Officer candidates are cast in the role of 
2nd lieutenants, lieutenants as company commanders, and 
senior NCOs as platoon leaders. In addition, a battery of 
psychological tests are used to measure attitudes, mental 
ability, reading skills and interests. Peer ratings and 
self ratings combined with those of the trained assessment 
staff provide a trifocal view of each individual tested. 

As indicated by the USAIS definition, the term assessment 
center more properly refers to a process or program rather 
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than to a physical location. In fact, some "assessment centers" 
are taken to various corporation field sites for administration 
vice transporting candidates to an "assessment center" facility. 

B. HISTORY 

Various articles have traced the genesis of the assessment 

center to the Bible. Wilson and Tatge [70] reference the 

advice which Jethro gave to Moses on selecting his rulers: 

Moreover thou shalt provide out of all the people, able 
men, such as fear God, men of truth, hating covetousness; 
and place such over them to be ruler of thousands, and 
rulers of hundreds, rulers of fifties, and rulers of tens. 

Jaffee [35] cites Judges, Chapter 7 as the first use of 
situational techniques: 

. . . the Lord provided Gideon with a means of choosing the 

best among his men. "Bring them down unto the water, and 
I will try them for thee there .... Separate every one 
that lappeth of the water with his tongue, as a dog lappeth, 
him shalt thou set by himself; likewise every one that 
boweth down upon his knees to drink . . . ." 

From an initial force of 32,000 men, Gideon was able to 
select, by such techniques, an elite group of 300 and routed a 
numerically superior army. 

Prior to, and during, World War II the Germans used multiple 
assessment techniques in officer selection. The German efforts 
inspired the British War Officer Selection Boards (WOSB) , which 
in turn influenced the program of the Office of Strategic 
Services (OSS) in this country. The other primary influence 
in the OSS program was the presence of Dr. Henry Murray of 
Harvard who brought with him concepts derived from his multiple 
assessments in personality research [97]. Assessment of Men [99] 
by the OSS Assessment Staff is a detailed and very readable 



account. of this. effort. 
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All of the programs mentioned thus far were designed for 
the purpose of selection, however the wartime environment 
afforded neither time nor opportunity for proper validation. 

Following the war, only the British retained operational 
assessment programs. Their Civil Service Selection Boards 
(CISSB) began validation efforts, while the WOSB evolved into 
the Royal Commissions Board (RCB) of today. 

As West Germany began rebuilding her armed forces, the 
Federal German Navy established an extremely sophisticated six- 
week assessment program for its officers [57]. This program 
has recently been merged into a tri-service staff officer 

course . 

* 

Since the close of the OSS centers, the U. S. military 
has only dabbled sporadically in assessment center research. 
The latest effort was launched in 1973 when the U. S. Army 
chartered a one-year evaluation of a pilot personnel assessment 
program. This pilot program was established at the U. S. Army 
Infantry School (USAIS) , Fort Benning, Georgia. 

In addition to the British and Germans, the Australian 
Army and Israeli armed forces are known to have assessment 
programs . 

Although the U. S. military has lagged other nations in 

the use of assessment centers since WW II, U. S. industry has 

pioneered in the field. Dr. Douglas Bray of AT$T set up an 

assessment program within the Management Progress Study in 

1956. Dr. Bray [7] describes this study as follows: 

Although the Management Progress Study was instituted as a 
long-term study without any expectation of immediate 
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practical results . . . . , the mere conduct of the study 

and the reporting of gross observations on the total group 
of subjects have led to significant changes in certain 
personnel practices . . . Among them are .... the 
application of the assessment center method to the selection 
and development of managers. 

The Management Progress Study was designed as a ten-year 
longitudinal study--and indeed it was continued to fruition-- 
but the merits of the assessment center were quickly recognized 
and the first operational assessment center in the AT§T system 
opened its doors in 1958. Following AT&T's lead, other U. S. 
firms, such as : IBM, General Electric, J. C. Penney, Standard 

Oil (Ohio) and Sears, established assessment centers, and the 
program spread to other countries. 

The influence of the AT§T program has been considerable 
and most of the subsequent assessment programs bear at least 
partial resemblance to it. 

C. OVERVIEW 

Although all assessment programs emanate from the same 
basic concepts, they differ considerably along certain dimen- 
sions, notably: 

1. Ob j ectives (Selection, development, placement or research). 

Most of the applications are for selection, however a 

growing number have development as a secondary objective. 

2 . Program Philosophies 

Assessment philosophies range along a continuum from 
psychometric to behavioral - -or in another sense, from signs to 
samples. There are a few programs at either extreme, however 
most fall nearer the center and incorporate both signs and 
samples . 
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3 . Program Lengths 



These vary from one day to six weeks. The majority 
are five days; with two and one-half to three days of assess- 
ment followed by two to two and one -half days of evaluation 
and report writing by the assessors. 

4 . Assessment Variables 

Since most programs are selection-oriented, the big 
emphasis has been on stable variables. If development were 
a program objective, unstable variables would have to be included 
to identify those areas offering a viable training potential. 
Numbers of variables assessed vary from ten to fifty- two. 

Howard, in her survey of current assessment programs, [34] 
found the following dimensions to be the most popular: 

"(a) leadership, (b) organizing and planning, (c) decision 
making, (d) oral and written communications skills, (e) ini- 
tiative, (f) energy, (g) analytical ability, (h) resistance 
to stress, (i) use of delegation, ( j ) behavior flexibility, 

(k) human relations competence, (1) originality, (m) controlling, 
(n) self-direction, and (o) overall potential." 

5. Program Content (Instruments and techniques). 

The content of the program should be an extension of 
the program philosophy. The listing which follows will proceed 
from signs to samples and will be as inclusive as possible, 
vice representative of any single program: 

a. Paper and pencil tests of attitudes, interests, 
mental ability, and achievement (math, reading grammar, etc.). 

b. Projective personality tests such as sentence 
completion, adjective check-lists, etc.. 
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c. Interviews by assessors. Some programs also have 
assessees interview other role-playing assessees to evaluate 
their interviewing ability. 

d. Written assignments such as essays. These may be 
solicited prior to arrival at the center. The topic may be 
of the candidate's choice, or assigned to him. 

e. Simulations may be either individual (the in-basket 
for instance) or "group '(such as readeiTeSs "'group discussions, 
management games, war games, etc.). 

The in-basket is a widely-used individual simulation 
with high face validity and a concomitant training effect. It 
consists of a series of notes, memos, and letters typical of 
the in-basket of the position for which the candidate is being 
evaluated. The scenario imposes a time constraint and forces 
the assessee to work alone without recourse to other individuals. 
He is provided ample information on the role he will play, (such 
as an organizational chart, pertinent personal data on key 
associates, etc.) and is asked to put all of his responses in 
writing. He may delegate items to subordinates, take action 
himself, defer action until further information is obtained, 
schedule meetings, or write letters, as he sees fit. Most 
programs schedule an interview session following the in-basket 
where an assessor can question the candidate on the motives 
behind his actions. Evaluations of the in-basket range from 
informal subjective impressions to highly-s tructured objective 
analyses . 

Leaderless group discussions typically assign the 
candidates differing points of view which they are to defend 
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individually. Following this discussion, they are asked to 
rank-order the positions. Since the points of view are designed 
to be equal in desirability, the candidate who is most per- 
suasive, influential and accepted by the group should emerge 
with the "winning" position. 

The various "games" used in assessment programs 
can provide evaluations of leadership ability, interpersonal 
skills , 'decision 'moking^h*i'l ity, - organizational .ability and 
resistance to stress - -depending on the particular objectives 
of the program. 

6 . The Assessors 

Several sources state that familiarity with the position 
sought by the candidates is the single most important attribute 
of an assessor. This probably explains why most programs uti- 
lize line managers two to three levels above the "assessment 
position" as assessors. A few programs employ clinical psy- 
chologists to interpret the projective tests, but very few have 
psychologists actively involved in the assessment and evaluation 
processes. Assessor/assessee ratios range from 1:1 to 1:4 and 
length of assignment as an assessor varies from one program 
(one to two weeks) to six to eight months or longer. Assessor 
training periods may be as short as a few days or as long as 
two to three weeks. The training may include lectures, films, 
etc., but the most effective and widely-used training technique 
is to involve the trainee in actual assessments on a "norm 
group." AT$T uses successful incumbents of the position being 
sought by candidates as the "norm group." This provides assessor 
trainees with two benefits: (1) a tangible example of performance 
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by organizationally defined "successes," and (2) an opportunity 
to err while learning without jeopardizing a candidate's career. 

A point upon which there is unanimous agreement is that well- 
trained assessors are an absolute prerequisite of an effective 
assessment program. 

7 . The Evaluations 

After the assessments are completed, the assessors 
document their -xd/servations . A^mee-tang -'is ^then-hold where can- 
didates' records are reviewed individually and ratings made 
by the assessors on the basis of all available information. 

Any disparities in the ratings are discussed further, however 
few organizations require consensus (e.g., that disparate 
ratings be changed) . The final result may be "pass-fail" recom- 
mendations to line management, or merely reports of the assessors' 
evaluations. These reports are usually in narrative form (con- 
sidered best for use by laymen) , however some firms prefer 
profiles consisting of graphs or scales. 

8 . Uses of the Data 

Most programs are selection oriented, however final 
selection decisions are not made by the assessors (with the 
possible exception of initial hiring situations where job 
performance data is not available, and the Federal German Navy 
program). In most programs, assessment evaluations are furn- 
ished to line management where it can be integrated with other 
information, such as performance appraisal data, as the managers 
see fit. In such cases (the majority) selection decisions lie 
solely with line management. 
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Development applications are few, but increasing in 



numbers. . They generally provide the candidate with knowledge 
of his strengths and weaknesses. The "ultimate" outcome would 
be a comprehensive development plan specifically tailored to 
the individual. 

Programs "with placement as an objective are rare. The 
Sohio program includes placement as an objective and is described 
by ^Pinkie- -and Jones [85]* • . 

9 . Feedback to Candidates 

Feedback is a part of most programs - -including selection 
oriented ones- -even though it represents a developmental function. 
It may be provided to all candidates, or only to those who 
request it. Feedback may be given by lay assessors, psychologists 
or line managers- -as the organization prefers. It is usually 
oral, confined to observed behaviors and test performance, and 
provides the candidate with recommendations for a personal 
development plan. 

10 . Summary 

Bender [3] provides an excellent overview summary. He 
conducted a survey of organizations operating assessment centers 
and obtained data from thirty-four operational programs. The 
results of his survey are presented in the folloxvring figures 
and provide a good picture of the current character of assess- 
ment in the United States and Canada. 
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Figure 1. 



Primary Evaluation Devices Used by Companies 
Operating Assessment Centers 



In-basket exercises used. 

Business. games exercises used. 
Assigned roles 
Nonassigned roles 

Leaderless group discussion exercises 
used . 

Assigned roles 
Nonassigned roles 

Films used. 

Video tapes used. 

Psychological tests used. 

For the most part, assessment devices 
are locally produced. 

For the most part, assessment devices 
arc purchased externally. 

An in-depth background interview is 
given to assessees. 

Source: Bender [3] 
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Operating Procedures Followed in Administration 
of Assessment Centers 
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23 




Re-assessment of assessees is permitted. 
A checklist for evaluation is used by 


19 


15 




assessors during evaluation. 


24 


10 




Peer evaluations are made by assessees. 
Assessment center has been validated 


24 


10 




against performance criteria. 


16 


18 





*Two companies considering for future evaluation. 

**Only one organization indicated that it had not yet assessed 
either females or minorities, but it added that no restrictions 
existed for not doing so. 

Source: Bender [3] 
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Figure 3. 

Operating Characteristics of Assessment Centers 



Number of assessees evaluated when your assessment center operates: 



_6 8 1£ 12 18 24_ 

Response frequency 13 2 2 11 2 3 

Levels assessed '’-by "your asscssment^cente-r : * 

First Line Middle management Upper management Newly hired 

24 17 3 7 

Number of days your assessment center operation takes: 

1 1^5 2 2Jj_ 3 3^ 4 _5 6 

Response frequency 1 2 Til 1 T 1 T 11 1 

Total number of evaluation exercises used during the operation 
of your assessment center: 



A^678910^:U154J) 
Response frequency 763411 2 2 1 1 

Number of years your assessment center has been in operation: 



0-1 i 2 34 ^ i M ^ 

Response frequency: 1 12 731231 1 1 

Number of employees you have assessed to date:+ 

0 0-500 500-1000 1000-3000 75000 

Response frequency: 1 25 5 2 1 

*Several companies indicated that their assessment center was 
used for evaluation of more than one level of supervision. 

+One center was not yet operational. 

Source: Bender [3] 
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Figure 4. 



Evaluation Parameters Most Frequently Observed in 

Assessment Centers 



Parameter 



Impact 

Energy 

Eoxc e fulness 

Perception 

Creativity 

Range of interests 

Salesmanship 

Listening skill 

Risk-taking 

Independence 

Management control 

Use of delegation 

Problem analysis 

Leadership 

Oral communication skills 
Oral presentation skills 
Written communication skills 
Organizing and planning 
Decision-making skills 
Inner work standards 
Resistance to stress 
Attitude toward peers 
Attitude toward subordinates 
Attitude toward superiors 
Self-evaluation 
Behavioral flexibility 

Source: Bender [3] 



Response Frequency 

18 

21 

19 

16 

17 

12 

16 

12 

16 

14 

16 

21 

24 

29 

30 
21 

23 
27 

25 
10 

24 
12 

8 

11 

11 

21 
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Figure 5. 



Uses Made of Performance 
Generated by Assessment 


Data 

Centers 






Assessees are provided immediate 


Yes 


No 


Not 

Checked 


feedback on their performance. 
Feedback is given orally to the 


21 


12 


1 


assessee . 

Feedback is given in writing to 


33 


0 


1 


the assessee. 

Feedback is given through the line 


10 


22 


2 


organization . 

Feedback is given assessee only by 


16 


15 


3 


assessment center personnel. 
Assessee's evaluation is made 


21 


12 


1 


available to top management. 
Evaluation results are used to 

prepare a formal plan of develop- 


24 


8 


2 


ment for the assessee. 

A follow-up of an assesse's continued 
development after assessment results 
are reported is a routine function of 


23 


9 


2 


assessment center activities. 
Source: Bender [3] 


14 


17 


3 
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Figure 6. 

A Summary of Key Characteristics of Assessment Centers 



(with both assigned and 

/ 

with simulated 



In-basket exercises are used. 

Games and leaderless group discussions 
nonassigned roles) are used. 

Psychological tests are used in conjunction 
exercises . 

Assessment instruments are locally produced. 

In-depth interviews are conducted. 

Assessees are nominated by supervision. 

Four simulation exercises are used during evaluation. 

Assessment centers are operated by part-time directors. 

Females are assessed. 

Minorities are assessed. 

Homework is assigned. 

Assessment is conducted away from the work environment. 
Assessors rotate in viewing assessees during evaluation. 

permitted. 

are made by assessees. 
one assessee is used, 
people are assessed at one time. 

Assessors are two organizational levels above assessees. 
Assessment centers operate for three days. 

Assessee's performance is rated, not ranked. 

Rating systems are based on research. 

Parameters assessed depend on the purpose of the assessment 
center. 

Assessees receive immediate oral feedback on their performance. 
Assessees receive written feedback on their performance. 
Feedback is disseminated by assessment center personnel. 
Assessment center results are used for formal development 
plans . 



Re-assessment is 
Peer evaluations 
One assessor for 
A maximum of six 



Source: Bender [3] 
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III. APPROACH 



A. INTERVIEWS 

Working within time and fund constraints, four sites were 
visited. These included two military (U. S. Army) applications, 
an AT§T (Pacific Bell Telephone) assessment center and an 
assessor training session for a one-day behaviorally-oriented 
program. A brief summary of the visits with lists of signifi- 
cant aspects of each site and application follows. 

1 . United States Army War College (USAWC) , Carlisle Barracks , 

Pennsylvania 

This visit afforded the opportunity to interview indi- 
viduals who were involved in the USAWC "Study of Leadership for 
the Professional Soldier" [128] , [129] , [130] , and [131] ; namely, 

Dr. Donald D. Penner and LTCOL Dandridge M. Malone. In addition 
to providing copies of the references cited above, and amplifying 
information on the study, they outlined some of the Army's other 
activities which preceded the USAIS assessment program. 

The influence of the Center for Creative Leadership in 
Greensboro, North Carolina (Olmstead, et . al . [12 2] refers) was 
explained. When the Army became seriously interested in assess- 
ment centers, a few years ago, twelve Brigadier Generals were 
sent through the Center for Creative Leadership to determine 
their reaction to an assessment program. This program was 
purported to contain several paper and pencil tests and one 
complex simulation. The simulation is of particular interest. 

It featured a scenario set in the 31st century, and required 
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candidates to cycle through six defined leadership roles. The 
futuristic setting was chosen to eliminate any possibility of 
bias from previous candidate experience. The entire simulation 
was observed by assessors through "one-way" mirrors', and also 
video-taped. At intervals, action was halted and candidates 
were asked to record their perceptions of the exercise at that 
point. This simulation lasted an entire day, and was followed 
by a "detailed'* 'feedback which » stressed*. the -.comparison between 
the individual's self -perception and the perception of him by 
others . 

Because this program is oriented toward research and 
self -development , the Center does not provide any assessment 
information to a candidate's parent organization. It does, 
however, provide transcripts, video tapes and other data 
obtained, directly to the candidate. The twelve Brigadiers 
who underwent assessment were reported as enthusiastic in their 
endorsement of the program as a vehicle of personal inventory 
and development. 

LTCOL Quay Snyder [124] offered much detail on the 
AWC Inventory and Assessment. His review of relevant instru- 
ments and information on source material was especially valuable. 

Probably the greatest benefit from the AWC visit derived 
from the constructive criticisms of this writer's concepts of 
an assessment program, and the alternative suggestions offered. 
Though too numerous to acknowledge individually, there were 
many seeds which took root and flourished during the ensuing 
research period. The results provide much of the substance 
of this thesis. 
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Some of the more significant factors we re: 

a. The concept of more than one assessment during the 
course of a career. 

b. The "systems approach" to personnel management 
which views assessment centers as a subsystem of the larger 
system. 

c. The USAWC Leadership Study. 

-d. ..An ..assessment program .applied in ..a^ military environ- 
ment . 

e. An assessment program devoted exclusively to develop- 
ment . 

2 . United States Army Infantry School (USAIS) Assessment 

Center Pilot Project 

This operation represents a current military application 
of the "AT§T model," e.g., a program which utilizes a juix of 
paper and pencil tests and experiential instruments. The visit 
afforded interviews with the director, COL Wallace Veaudry; 
assistant director LTCOL Kenneth Smith; staff members, MAJ John 
Campbell and MAJ Richard Davis; and Dr. Kay Smith of the Army 
Research Institute (ARI) Field Unit which is co-located with 
the assessment center. 

The USAIS assessment program entails forty hours of 
assessment in a three-day period. This heavy schedule is a 
result of the inclusion of "competing" instruments for evalu- 
ation. At the completion of the evaluation, the program will 
be factor analyzed and those ins truments with little or no 
unique contribution will be eliminated. This should stream- 
line the program to a more realistic schedule. 
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A unique feature of this visit was the fact that 
current staff members had been involved in the design and 
implementation as well as the operation of the assessment 
program. Because of this, staff members were exceptionally- 
understanding and knowledgeable concerning the focus and 
requirements of this study. Additional background on U. S. 
Army assessment center research was provided, and an assess- 
ment center research program conducted at Fort McClellan, 

4 

Alabama circa 1965 was reported. It was described as a very 
expensive operation where candidates were assessed one at a 
time, thus the assessor/candidate ratio was something like 
12:1. In contrast, the present USAIS ratio is about 11:18, 
which is more representative of ratios in current industrial 
assessment programs. The use of video tape, and the structure 
and content of the counseling feedback were other facets of 
the program worthy of mention. 

In summary, the following items were of particular 
interest: 

a. Information concerning assessment center design, 
and implementation- -as well as operation. (Olmstead et . al . 
[122] pertains to design of this program). 

b. Simulations tailored to military situations. 

c. Use of video-tape in analysis and counseling. 

d. Feedback techniques and counseling training. 

e. Differences between programs, reflecting different 
levels of candidates and positions being assessed for, e.g., 
differences in course content between the following: 
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Course 



Candidate Level 



Officer candidate course E5 

Advanced NCO course E6-E7 

Basic Officer course LT 

Advanced officer course CPT 

f. Assessor training and employment. (Assessors are 
assigned for a normal tour of duty- -about two years - -therefore 
are long-term in contrast with short-term assessors prevalent 
in indust ry .-programs) . 

3 . Dr. Cabot L. Jaffee, Florida Technological University , 

Orlando, Florida 

The visit coincided with an assessor training session 
which Dr. Jaffee was conducting for selected civil service 
personnel of the Naval Training Center, Orlando. The training 
was in preparation for a one-day assessment program composed 
of behavioral instruments (simulations, etc.). This provided 
an opportunity to observe assessor training which included the 
administration and scoring of the in-basket exercise included 
in the assessment program. 

Dr. Jaffee has extensive experience in the field of 
assessment centers and has contributed much to the literature 
[16], [35], [36], [89] and [90]. Because of this, his comments, 

criticisms and suggestions were particularly valued. 

Highlights of the visit were: 

a. Observation of assessor training. 

b. Views of a prominent consultant in the field of 
assessment centers. 
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4 . Personnel Assessment Review (PAR) Program, Pacific 

Telephone Company, Oakland, California 

Because of the great influence of the "AT§T model" on 
subsequent assessment programs, a visit to a Bell system program 
was given high priority. During the interview attention was 
focused on operational aspects rather than design or theory. 

Salient points of the program included: 

a. Clinical combination of variables in the evaluation 

process . 

b. Practice of having candidates' supervisors observe 
the evaluation process. 

c. Pre-assessment interviews done by assessors at 
candidate's organization. 

d. Feedback to candidates done by assessors at candi- 
date's organization. 

e. Practice of mixing men and women in groups vice 
assessing them separately. 

B. LITERATURE REVIEW 
1 . Overviews 

This section is provided as a guide for the reader who 
seeks additional overview information. It does not contain 
descriptive data, per se, but rather directs the reader to 
selected articles in the literature. Those readers not 
desiring such referral advice may proceed directly to section 
III-B-2, Current Applications . 

Several articles in the literature provide excellent 
overviews. These will be cited in the following order: (1) 

General articles covering a wide range of programs, (2) Articles 
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dealing with a particular type of program (e.g., the five-day 
program patterned after the AT£jT model, etc.), (3) Articles 
dealing with specific organizational programs. 

An article in Business Week [68] provides a broad 
survey of the field from one-day assessment programs, to the 
"five-day" programs employed by larger firms such as ATfjT, 

J. C. Penney, Standard Oil (Ohio), etc. and gives some rough 
cost figures. - Howard .-[34 ] presents -an - excellent aggregation 
of information from the entire field. Cohen and Jaffee [16], 
although primarily concerned with government applications, 
cover most of the significant developments in the field. They 
also provide some predictions on future growth and changes in 
government assessment center applications. 

Slevin [59] describes a program typical of those found 
in the major firms employing assessment centers (e.g., AT§T, 
Sears, IBM, General Electric, J. C. Penney, Standard Oil of 
Ohio, etc.). This article has a particularly comprehensive 
coverage of the subject, touching briefly on most of the 
major topical areas. This writer considers Slevin 's predic- 
tions for the future of assessment centers to be particularly 
perceptive and well-founded. Other authors writing on assess- 
ment programs of this type include: Albrook [1], Byham [11], 

and Byham and Pentecost [12] . 

McConnell [43] describes a one day assessment program 
in a small company. McConnell [44], [45] and McConnell and 

Parker [46] discuss the one day "package" assessment program 
available from the American Management Association (AMA) . 
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Information on the AT§T (Bell System) programs is 
abundant in the literature. Articles providing general data 
include Jaffee [35], Jelks [37], and Wikstrom [69]. 

The Standard Oil of Ohio (Sohio) program is treated 
in detail in the book by Finkle and Jones [85] . Chapter four 
of the book contains the overview. This program is also 
described by Hardesty and Jones [31]. 

Veaudry and Campbell [65] describe the USAIS assess- 
ment center pilot project, while Snyder [124] covers the USAWC 
developmental assessment program. Information on foreign 
military programs are found in the USAIS unpublished report 
of a trip to the RCB [126] and Schmahling's [57] article on 
the Federal German Navy’s program. 

2 . Current Applications 



a. Industry 

It is difficult to obtain an accurate estimate of 

the number of assessment centers in existence, however the 

literature provides a picture of increasing growth in this 

area. Pomerleau [53] notes: 

. . . AT§T affiliates . . . currently operate over 50 

centers throughout the country processing upwards of 
10,000 candidates a year at an annual cost of $5 million 
. . . contributors to professional journals ... in the 

last five years, have written some fifty-four articles 
and reports describing and evaluating the assessment 
center method. 

McConnell [45] describes the record of a one-day 
package assessment program: 

In 19.70, at the end of a five year research and develop- 
ment project, the American Management Association (AMA) 
introduced a multimedia assessment centre programme . . . 

in the three years since its original publication, over 
150 organizations have used it for assessment of their 
management development needs. 
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Byham [11] offered this estimate of assessment 
center growth, circa 1970: 

To my knowledge, twenty companies have been responsible 
for assessing more than 70,000 candidates in the last 
ten years, but at least 100 more companies are developing 
centers or are in an advanced stage of center planning. 

Many others are "looking into the idea." As an indication 
of this interest, I might cite the fact that more than 
200 company representatives attended conferences on the 
assessment center method during 1969. 

Two articles have been selected to provide detail 

on current industrial assessment programs. The first concerns 

a one-day "package program" offered by the American Management 

Association (AMA) . Such a program puts assessment techniques 

within reach of smaller companies who cannot afford to develop 

and maintain programs of their own. The article (by McConnell 

[45]) describes exportation of the AMA program to Europe and 

is written with a British flavor: 

The assessment centre programme consists of 5 basic parts: 

Chairman's training--A representative of the organization 
spends one week at an intensive training programme conduc- 
ted by Management Centre Europe to obtain all of the know- 
ledge and skills necessary to conduct all phases of the 
programme within his own organization. 

Assessor training- -Upon returning to his organization, the 
trained chairman selects five existing managers to be 
trained as evaluators in the programme. These managers, 
once trained, are referred to as assessors. Their training 
consists of a 28 hour multimedia. training course in which 
they are taught the. skills and techniques necessary to 
serve successfully as assessors. Included in this training 
is a measuring device to insure their skills are at a level 
at which they can successfully serve as assessors. 

Management simulation workshop- -Once the assessors are 
trained, a management simulation workshop is conducted. 

This is one day in length and consists of eight exercises. 
Twelve participants are selected to go through the eight 
exercises and are observed in these exercises by the 
assessors. The assessors' role in the workshop is to 
observe and record the behavior of the participants. fol- 
lowing the one day workshop, the participants are allowed 
to return to their jobs. 
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Evaluation meeting - -The assessors and the chairman meet 
following the management simulation workshop to report 
and evaluate the behavior of each individual observed. 

This final evaluation yields the report on each individual 
participant in 12 specific management abilities, overall 
management ability, and the individual's organizational 
potential. This evaluation meeting requires approximately 
12 hours. 

Feed-back- -At a feed-back session between the chairman 
and each participant, the results of the centre are com- 
municated to the participant, compared with his own self 
evaluation, and specific development plans for the par- 
ticipant are established. 

The second article (Byham [11]) describes the most 
widely used format: the five-day program patterned after the 

AT§T model. Although the following material was based on the 
J. C. Penney program, it is generalized enough to typify 
practices of many of the major firms. 

Sunday 

Six management assessors meet at a conveniently located 
motel and organize materials for the week's activities. 

Late in the day, twelve candidates, all of them of com- 
parable rank in the company, arrive and settle in. 

Monday Morning 

Period 1: After orientation announcements the candidates 

are divided into teams of four, for participation in a 
management game. Each team is given a limited amount of 
capital to purchase raw materials, make a product, and 
sell it. The raw materials are usually tinker-toy parts 
which can be assembled into a variety of products of dif- 
ferent complexity, each of which has a different, pre- 
specified market value. 

The players must first decide how to invest their 
capital to maximize profits and then organize the pur- 
chasing, manufacturing, and selling operations. Assessors 
observe the players for signs of leadership, organizational 
ability, financial acumen, quickness of thinking, and 
efficiency under stress. 

Suddenly the players are notified that the prices of 
the raw materials and the products have been radically 
changed, requiring drastic redeployment of capital and 
extensive operational reorganization. As soon as they 
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have regrouped, these prices are abruptly changed again. 

The actions the players take allow the assessors to esti- 
mate their adaptability. 

The game is then halted, and each candidate is asked 
to write a report evaluating his own performance and that 
of his fellow players. 

Period 2: The candidates are divided into groups of six. 

While one group takes written psychological tests, the 
members of the other group are interviewed individually 
by the assessors. The assessors have been provided with 
detailed background information on each man, and they use 
this to probe for evidence of drive, motivation, and sense 
of -self -development . This Assessment Interview, so called, 
is ordinarily the only exercise in the assessment process 
that focuses on the candidate's past behavior. 

Monday Afternoon 

Period 1: The testing and interviewing groups are reversed. 

Period 2: In two leaderless groups of six, the candidates 

join in discussion of a promotion decision. Here the can- 
didates play the role of supervisors brought together on 
short notice by their boss to pick one man from a pool of 
six for advancement. Each candidate receives the file of 
one of the men in the pool, whom he is then to "champion" 
for the promotion. After each candidate has studied his 
protege's folder, the group meets for an hour's discussion 
to choose the man it will recommend. Assessors observe 
the candidates' exchanges in the meeting for signs of 
aggressiveness, persuasiveness, expository skill, energy, 
flexibility, self-confidence, and the like. 

Alternative exercise: In leaderless groups of six, candi- 

dates discuss the 20 most critical functions of a manager 
and list them in order of importance. (This forces them 
to think about the qualities on which they are being 
assessed.) Each group then chooses a spokesman who pre- 
sents the list and the rationale behind it to the whole 
group of assessors and candidates. 

Monday Evening 

Each candidate receives material on how best to conduct 
employment interviews and also the resume of a job appli- 
cant. He studies these for use in one of the exercises 
on the following day. He may also receive special phone 
calls--for example, the Irate Customer Phone Call. 
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Tuesday Morning 

Tuesday morning is devoted to the In-Basket Exercise. This 
simulates the experience a candidate would have if he were 
suddenly and unexpectedly promoted a grade or two and 
arrived at work one morning to find his in-basket full of 
unfamiliar material typical of the sort he would then have 
to. handle. Me is instructed to go through this material 
and deal with the problems, answer the inquiries, request 
additional information where he needs it, delegate tasks 
to proper subordinates, and generally organize and plan 
just as he would if he had actually been promoted. 

Tuesday Afternoon 

Period 1: The candidates conduct the employment interviews 

for which they prepared the night before, each interview 
taking place in the presence of an assessor. The appli- 
cants are college students who have been especially trained 
in the applicant's role. The interview itself lasts 
roughly half an hour, after which the applicant leaves and 
the assessor quizzes the candidate to determine what in- 
sights he has obtained about the applicant. 

Period 2: The next exercise is the resolution of disci- 

plinary cases. In groups of four, candidates decide how 
to allocate their time between three such cases and then 
decide the cases themselves, within one hour. This exer- 
cise provides the assessors with information on a candi- 
date's appreciation of personnel problems and his 
sensitivity to subordinates' views of events and actions, 
as well as insight into his behavior within a group. 

Alternative exercise: The candidates are assigned roles 

as city councilmen who meet to allocate a $1 million 
federal grant to the city departments. Each "councilman" 
interprets a briefing document provided by a city agency- - 
the police department, sanitation department, water depart- 
ment, and so on--and tries to get as much of the grant 
allocated to this agency as possible. Again, effective 
discussion is limited to one hour. 

Tuesday Evening 

Detailed data on a company was provided to all the candi- 
dates. Each is asked to examine its financial and 
marketing situation from the viewpoint of a consultant 
and to prepare a written recommendation for its board of 
directors on the future expansion of a particular part 
of its product line. 

At the same time, also in preparation for the next 
day's activities, the assessors study the results of the 
candidates' In-Basket tests in detail. 
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Wednesday 

Period 1: Four groups are formed, each consisting of three 

candidates and an assessor. Each candidate takes his turn 
presenting his oral analysis of the company data studied 
the night before and submitting written recommendations. 

Period 2: These three candidates work together for an 

hour to reconcile and consolidate the recommendations. 

Period 3: The In-Basket Interview follows, in which an 

assessor discusses with a candidate the various actions 
he took. This further defines each man's grasp of typical 
problems and opportunities. 

Wednesday Afternoon 

In a final group session, the candidates rate each other 
and ask any . questions they may have. They then leave for 
home . 

Wednesday Afternoon to Friday 

The assessors discuss the candidates and prepare their 
ratings and reports. 

Throughout all the exercises, the assessors have been 
rotated so that as many as possible have had a chance to 
observe each candidate closely. Thus, in these discussions, 
the assessor who conducted Jones's personal interview 
summarizes his background and his own impressions of his 
behavior in the interview; next the assessor who checked 
what Jones did in his In-Basket Exercise and interviewed 
him on it presents his impressions; and so on. Each 
assessor attempts to keep these descriptions nonevaluative 
and objective. 

Only when all the assessors who have observed Jones 
have spoken does the group begin to judge his behavior 
from the viewpoint of his management potential and the 
directions in which he needs to develop. After they have 
reached a consensus, they prepare a final report. 

Within two weeks a manager who has had experience as 
an assessor meets with Jones to communicate the results. 

In this meeting he lays stress on the areas in which Jones 
needs to develop himself and encourages him to set appro- 
priate goals. 
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b. Military 

Four sources have been selected to provide samples 
of widely divergent military programs. The first, Snyder [124] 
describes Inventory and Assessment at the United States Army 
War College (AWC) . Although this program is atypical of the 
assessment programs described thus far, it is of interest 
because it is purely development-oriented. Instead of a tightly 
scheduled program of a few days duration, the assessment is 
interspersed throughout the academic year (September - June) . 
Another difference is that faculty coaches or counselors replace 
the assessors previously described. Each coach is assigned 15 
students to counsel throughout the school year. The program 
employs five assessment techniques, e.g., Baseline, Inventory, 
Optional Personal Inventory, Professional Self-Assessment, 
Experiential Assessment, and Value-Gained Inventory. They are 
described as follows: 

(1) Baseline Inventory . A questionnaire is mailed 
to the student prior to his arrival at the AWC. The student 
indicates his competence in a variety of areas related to the 
curriculum and returns the questionnaire, whereupon the results 
are compiled. Feedback is provided which shows the class range 
and mean on each item. In this way, the student and the school 
can tailor individual curricula to best advantage. Future 
plans are to include additional instruments in this phase, such 
as: the Graduate Record Exam, to assess academic potential, 

and other .tests for reading comprehension, listening compre- 
hension, writing skills, etc. 
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(2) Optional Personal Inventory (0P1) . Research 
on executive development is said to identify a need for 



reassessment of personal and professional goals which occurs 
when an individual is about age forty and two -thirds of the 
way through his career. The OPI was designed to meet this 
need. A battery of tests is available to the student which 
can be administered and analyzed by his faculty coach. The 
battery includes .-measures of speaking, reading and writing 
abilities, attitudes, personality traits, needs, vocational 
interests , etc . . 

(3) Professional Self-Assessment . This item 

provides feedback in three ways: first, to the student, second 

to the instructor, and third to the institution. The need for 
such a feedback system is underscored by the fact that the 

AWC does not utilize an academic grading system. The self- 
assessment consists of a final examination administered to the 
students after which they are provided with solutions to enable 
self-grading . The unsigned exam papers are then given to the 
instructor for analysis by him and the institution. 

(4) Experiential Assessment . A group problem- 
solving exercise is conducted by a team of students lead by 
a faculty coach. Upon completion, exercise performance is 
critiqued by a panel of experts. Each student is provided 
feedback on his performance by the faculty team-leader, his 
peers and the panel of experts. 
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(5} V alue -Gained Inventory . Essentially, admin- 
istration of alternate, but equivalent, forms of the Baseline 



Inventory at the end of the academic year. Statistical analysis 
would be utilized to determine statistically significant changes 
Results could then be provided to students and faculty, and 
also used in curriculum development. 

The AWC program was initiated in academic year 
1973 - 1974, therefore it is too early, for an evaluation, how- 
ever,. the implications for the Naval Postgraduate School (NPS) 
are obvious and would suggest a close and continuing follow-up. 

The second military application to be considered 
is the pilot assessment center project at the United States 
Army Infantry School (USAIS) , Fort' Benning, as reported by 
Veaudry and Campbell [65] . This program is patterned after 
the "AT§T model" and entails three days of intensive assess- 
ment. There are actually three separate programs: one for 

senior noncommissioned officers (NCOs), a second for lieutenants 
and a third for captains, however, they follow the same basic 
philosophy and share many of the commercial instruments, e.g., 
paper and pencil tests, inventories, etc.. The programs are 
based on the "Leadership for Professionals" study conducted 
by the Army War College. Ten exercises were designed for 
each program, and combined with batteries of paper and pencil 
tests. The resulting assessments include writing exercises, 
in-baskets, leaderless group discussions, war games, field 
exercises, simulations of organizational problems in the 
field and three types of interviews. 
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Evaluation reports consist of numerical ratings 
and narrative profiles describing candidates' strengths and 
weaknesses. The narrative profiles include a one page summary 
of abilities and an eight to twelve page detailed description 
based on the observations of the assessors. 

The pilot assessment center project was chartered 

for one year and given the following missions: 

Test the assessment concept as a leadership and career 
development technique. 

Test the assessment concept as a precommission screening 
device . 

Determine the feasibility and applicability for Armywide 
use of the concept and techniques developed under the first 
two steps. 

As with the AWC program, the assessment center project has 
significance for the Navy and warrants close scrutiny. 

The British Royal Commissions Board (RGB) is one 
of the oldest military assessment programs in continuous oper- 
ation. The USAIS Assessment Center Project trip report [126] 
provides the following overview of the RCB: 

The RCB . . . has the mission of selecting candidates 

with the potential qualities of character, ability and leader- 
ship necessary for a career as an officer. All candidates 
for either Short Service, Limited Service or Regular Com- 
missions must be evaluated by the RCB prior to any pre- 
commissioning training (Sandhurst or University course) . 

The RCB has a President (Major General), three Vice 
Presidents (Brigadiers) , a Training Officer (LTC) , (and 
twelve assessors) (six Lieutenant Colonels and six Majors), 
all assigned on two year tours. The Board has the capa- 
bility of assessing 48 candidates each week, organized 
into 6 groups of 8 each. Each group, therefore, is directly 
supervised by a Major, whose principal function is detailed 
observing, and a LTC, whose functions include observing and 
conducting in-depth interviews of each of the 8 candidates. 

Although there is no permanent psychologist assigned 
to the Board, the Board draws on the professional advice 
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of psychologists assigned to the Army Personnel Service, 
and a civilian firm of consultants. In addition, each 
group of candidates receives an educational/intelligence 
rating from an Education Advisor attached to the RCB from 
the Army Education Board. 

The criteria used for evaluating each candidate are 
(1) Training Potential, (2) Applied Ability, and (3) 
Character. Within each major criterion, the following 
facets are considered: 



( 1 ) Training Potential 

(a) Officer intelligence rating (IQ) 

(b) Educational standard 

(c) Self-expression (written and oral) . 



The Training Potential is assessed by the Education 
Advisor, based on records (achievement vs. opportunities) 
and tests conducted at the RCB. The candidate's overall 
rating is given a numerical value as follows: 



Strong 


Good 


Adequate 


Limited 


Weak 


10,9 


8 


7 


6 


5,4, 


(2) 


Applied 


Ability 







(a) Planning ability 

(b) Practical ability 

(c) Physical ability. 



(3) Character 



(a) Coolness (ability to relax) 

(b) Initiative 

(c) Sense of urgency 

(d) Determination 

(e) Military compatibility 

(f) Sense of responsibility 

(g) Sense of awareness (breadth of knowledge) 

00 Liveliness 

(i) Dominance 

(j) Quality of character 

(k) Range of personal relations (ability to mix) . 

The schedule for each session is divided into Parts I 
and II. Due to the large number of candidates, it was felt 
necessary to establish an initial screening process to 
reduce the number of candidates down to 48 for the actual 
assessment. The schedules are as follows: 
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PART I 



Monday 



Tuesday 



Intelligence test 
Current affairs 
Essay 
Interviews 

Evaluation of reports 



Part I Board sits 
Eliminated depart 



PART II 



Tuesday 



Wednesday 



Thursday 



Group discussion 



Written projects 
Individual tasks 
Obstacle course 
Lecturettes 



Group race 
Final Boards 
Results posted 



Group task 
Group race 
Interviews 



The Majors and ETC's are rotated each session to prevent 
bias, and their recording of information is done independently. 
The Major simply observes; he does no interviewing, neither 
does he see any written reports or references on the candi- 
dates. In addition to observing^ the LTC conducts interviews 
in depth, but also does not see any reports or references. 

A preliminary finding is determined after the first day with 
the Brigadier reconciling the differences between the Major 
and the LTC. The reports and references have been previously 
studied and graded by the Brigadier. 

Based on the preliminary findings (Straight up, Doubtful 
up, Doubtful, Doubtful down, Straight down), the President 
of the Board will then conduct his own interviews, observe 
candidates as they (perform) executive leadership "tasks," 
and attend those final boards he feels are critical. The 
Brigadier makes the final, overall assessment rating, 
subject to the approval of the President. The rating is 
either Pass, Fail or Return; with copies forwarded to 
Sandhurst and the Ministry of Defense. Historically, the 
Board's pass rate has been some 19 percent for Sandhurst 
(Regular Commission) candidates, 11 percent for Limited 
Service candidates and 15 percent overall. 



cribed is the recently discontinued Staff Officer and Selection 
Course of the Federal German Navy described by Schmahling [57]. 
This six-week assessment program was in existence from February, 
1959 to December, 1973. During its fifteen years of operation, 
2287 officers underwent assessment. 



The final military assessment program to be des- 
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The basic objective of the program was selection 
of Staff Officers. In 1966, the selection category of Admiral 
Staff Officer was added. The criteria used consisted of three 
sets of variables pertaining to personality, mental ability and 
problem-solving capability. It became evident that personality 
traits were poor predictors and the program was finally based 
entirely on assessment of mental abilities. 

As of 1968, »an overall score has been obtained by 
combining scores from eight items, each with equal weight. 

These eight items include: four written tests, three essay- 

type exams, a term paper, four oral tests, two in-basket simu- 
lations, a war game and an oral presentation. During the years 
the program underwent constant refinement, and efforts were 
made to control for bias due to the differing backgrounds 
of the candidates. Tests were selected and designed to afford 
responses from different levels of candidate experience and 
sophistication. Emphasis was on methodology vice recall of 
previously learned knowledge. 

Despite the fact that the program was selection- 
oriented, a number of developmental effects were noted. For 
most officers, the return to academic pursuits after years of 
operational service was a challenge. Another stimulus was 
provided by the high standard of performance demanded and 
obtained which produced a wholesome competition between candi- 
dates. A third factor was the knowledge that assessors were 
ever-present and observing every nuance of the candidates' 
performance . 



4 7 



The only requirement cited for assessors was that 
one assessor for each group of ten candidates had to have com- 
pleted Admiral's Staff training. Thus, assessors typically 
possessed broad operational experience and were expected to 
have a sense of responsibility for their fellow officers and 
the capability of making pragmatic decisions. The practice of 
multiple assessment was seen as easing the burden of responsi- 
bility for the assessor and increasing the candidate's confi- 
dence in the final rating. 

With few exceptions, the candidate received only 
the final result upon completion of the course. Two reasons 
given for this practice were to save time and to prevent 
weak candidates from giving up midway through the program. 
(This writer does not consider either of these very compelling 
and speculates that the organization had other reasons for 
withholding feedback) . Schmahling asserts that the lack of 
feedback was rarely criticized but that many candidates felt 
that past performance evaluations should have been considered 
in the final rating. 

The Federal German Navy program has been replaced 
by a tri-service assessment center, however, as the author 
points out, the fifteen years of assessment experience is not 
lost, but can be profitably applied to the new assessment 
program . 

3 . Advantages (Support for the concept) 

Three articles have been selected which state the 
most commonly cited benefits of assessment centers . The first 
is by Ginsburg and Silverman [25] who described the design and 
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implementation of an assessment program for hospital personnel. 
They wrote as follows: 

Compared to other forms of personnel appraisal, this method 
is seen to be more effective because all assessees: (1) 
have an equal opportunity to display their talents (2) 
are seen under similar conditions in relevant situations 
designed to bring out the particular skills and abilities 
needed for the position or positions for which they are 
being considered, and (3) are evaluated by a team of 
trained assessors, unbiased by past association, who are 
intimately familiar with both the position requirements 
and the institutional climate ... 

The major contribution of the multiple assessment 
approach has been the use of situational tests or exercises 
. . . . the application of situational techniques to 

assessment has reduced the amount of inferences which must 
be made from the more loosely structured paper and pencil 
techniques. They provide more positive answers to the 
question, "Given these traits, how is he likely to behave 
in a work situation?" 

Situational methods also offer the potential of adding 
greatly to the scope of human characteristics which can be 
evaluated. Although more expensive and time-consuming to 
administer than usual appraisal procedures, the need to 
find ways of evaluating characteristics not covered by the 
latter is sufficient to warrant extensive experimentation 
with relatively elaborate techniques .... 

All in all, the center does seem to be predictive of 
managerial ability. Furthermore, a vital part of the 
Identification and Development Center is the action which 
will be taken to increase individual effectiveness. Tailored 
developmental programs will replace shot-gun attempts at 
training individuals for increased responsibility. 

Expanding on the benefits of development, Slevin [59] 



wrote : 



The assessment center can serve as a management develop- 
ment tool in two ways. First, the exercise of participating 
in a center is in itself a developmental experience. The 
individual gets practice in leadership skills, makes oral 
presentations, makes hypothetical management decisions and 
has them criticized, observes himself on a video tape 
replay of his performance and has other developmental 
experiences. Research has indicated that the video tape 
feedback alone has very promising potential for organizational 
development team building and other efforts (Weber [66]). 
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In addition, the whole process of the critical and intensive 
period of self insight that is experienced in a two to three 
day center is extremely constructive in providing the 
assessee with information on how to achieve positive change. 

He is provided with an opportunity to scrutinize his behavior 
in a critical manner not possible in on-the-job performance, 
and in a manner that might lead him to understand those 
behaviors that are most effective and least effective in 
his managerial capacity. 

Second, the assessment center is an excellent place for 
the formulation of a developmental plan. Executives serving 
as assessors are currently operating at levels in the organ- 
ization to which the assessee eventually aspires. They're 
in a position to evaluate the types of experience and 
training the assessee needs to reach their level of attain- 
ment. At no other time in the assessee' s experience with 
his organization will six upper management executives spend 
an intensive two to three hours discussing his strengths, 
weaknesses and developmental needs. At this point, in 
conjunction with the overall manpower needs of the organi- 
zation, a developmental plan may be engineered that is 
specific to the needs of this assessee. This makes obsolete 
the conception of a standardized developmental approach for 
all management and provides the unique capacity of a specif- 
ically engineered developmental program for each person 
assessed. 

The third article is by Byham [11] who summarized 
assessment center advantages as follows: 

Reports have proved to be remarkably valid. Longitudinal 
studies of thousands of employees assessed over the last few 
years indicate that this assessment method is much more 
accurate than traditional appraisal procedures, and these 
seem to be the reasons: 

*The exercises used are designed to bring out the specific 
skills and aptitudes needed in the position(s) for which 
a group of candidates is being assessed. 

(Note: This facet is commonly referred to as "combating 

the Peter Principle; since the candidate is evaluated on 
requirements of the position he is aspiring to- -not those he 
he has held in the past. When there is a large difference 
between these requirements- -e . g . , promotion from crafts to first- 
line management- - this factor becomes very significant). 

*Since the exercises are standar dized, assessors evaluate 
the candidates under relatively constant conditions and 
thus are able to make valid comparative judgments. 
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*The assessors usually do not know the candidates personally 
so, being emotionally disengaged, they arc unbiased. 

*The assessors are shielded from the many interruptions of 
normal working conditions and can pay full attention to the 
candidates' behavior in the exercises. 

*The procedures focus their attention on the primary kinds 
of behavior they ought to observe in evaluating a promotion 
candidate . 

*They have been trained to observe and evaluate these kinds 
of behavior. 

' Byham further points outcome indirect benefits which 
accrue from the center, e.g.: 

a. Candidate training 

b. Positive influence on morale and job expectations 

c. Subtle improvement of candidates' understanding 
of and attitudes toward organizational goals and 
policies 

d. Assessor training (said to be by far the most 
valuable "fringe benefit") . 

As a final argument, Byham tallies his survey of 
users of the centers: 

In a survey of the 20 companies that operated centers, 

I uncovered some 22 studies in all that showed assessment 
more effective than other approaches and only one that 
showed it exactly as effective as some other approaches. None 
showed it less effective. As I suggested before, these 
studies exhibit correlations between center predictions 
and achievement criteria such as advancement, salary grade, 
and performance ratings that range as high as .64. The 
companies appear satisfied that they are on the right track. 

Byham and Pentecost [12] echo most of the preceding 
with the following additional observations: 

a. ... participation in an assessment center stimulates 
the candidates self -development through self - ins ight . 

b. Appropriate training and development programs can be 
organized and, if practical, jobs can be developed to match 
the particular abilities of available candidates. 
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The report of a seminar conducted by the Foundation 



for Research on Human Behavior [112] illustrates the potential 
impact of face validity on group morale : 

It is important that the promotion of a given individual 
be not only the best selection but also that it appear to 
other candidates as a good selection. If a candidate is 
judged highly qualified mainly on the basis of tests, con- 
sultants' opinions, and other information relatively invisible 
to the candidate population, the advantages of promoting him 
have to be balanced against the possible disadvantages to the 
morale of the group. (However) if the candidates consider 
that the evaluation 'standards are clearly relevant to the 
job situation, they will tend to accept the results of the 
/ evaluations. 

Kraut and Scott [41] continue this theme: 

But face validity ... is high for assessment programs. 
Raters in an assessment program generally come away feeling 
that it does indeed work. The exercises seem reasonable, 
and one can make discriminations which appear meaningful. 

They also report results of a study which they conducted 
involving 1086 subjects. The results showed that assessment 
center ratings correlated highly with two major criteria: 

(1) promotion to the second level of management and (2) demo- 
tions from the first level of management. Both of the criteria 
were chosen to minimize contamination. Since the assessment 
objective was selection for promotion to first level management, 
the authors contended that the criteria selected were minimally 
contaminated by knowledge of assessment center ratings. 

Kraut [40] cites studies by Bray and Grant [10], Wollowick 
and McNamara [71] and Kraut and Scott [41] . In summary he 
states : 



These studies and many others lead one to conclude that 
assessment programs have validity in predicting those who 
will move ahead in an organization. Many of the studies have 
flaws, but there is a consistent pattern of apparent validity. 
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He also discusses a phenomenon reported by many studies: 
that the predictive validity of assessment center ratings 
improves as candidates are considered for promotions to second 
and third levels of management. He speculates on the reason 
for this : 

It may be that the performance of first-line managers 
in this company is heavily dependent on technical skills 
which the men carry into that level of management (first) 
and that the power of the assessment center shows up pri- 
-inarily when they are considered for higher levels of 
management . 

In reviewing a previous study by Hinrichs [32] , Kraut 
points out a possible key to the advantage of assessment centers 
over traditional techniques. He says that observed interpersonal 
activity was a major contribution in the assessment, but not 
in traditional selection procedures. 

In response to criticisms of "deciding a man’s career 
on two days of observation," he writes: "The assessment center 

may represent two days more of observation than existed before." 
Of course the original criticism is fallacious since most assess- 
ment results augment rather than supplant performance appraisals 
and other sources of information used in promotion decisions. 

Dodd [18] reports a study from which lie draws an 
interesting conclusion regarding assessment techniques and 
s tereo typing : 

The present study offers some hope that, far from sup- 
porting conformity or suppressing independence, tests vali- 
dated against assessment performance, when used to supplement 
supervisory recognition, could break the hold of conformity 
on present selection procedures and open management ranks 
to those who value independence. 

The undisputed landmark in the field of assessment 
center validation studies is the Management Progress udy 
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of AT$T. No article on the subject can be considered inclusive 
without paying homage to "The Study." Bray and Grant [10] offer 
the most definitive article on this study and it is augmented 
by a series of articles amplifying on it and providing details 
of other AT§T validation studies. 

Bray and Campbell [9] describe a study on the selection 
of salesmen which yielded a predictive validity of .51 for 
assessment center ratings correlated to job performance ratings 
obtained by special evaluation teams. 

Campbell and Bray [14] discuss criterion and contami- 
nation problems in validity studies. They cite two studies 
which attempted to overcome these. One was a Michigan Bell 
study which compared the first forty men assessed and promoted 
with the last forty men promoted before the assessment program 
began. The other. study, done in the New England Company, compared 
candidates rated "acceptable” with thoserated "not acceptable." 

In both of these studies the ratings of the assessment center 
correlated highly with subsequent performance and were superior 
to traditional selection procedures. The authors make a final 
important point that predictive validity alone is not sufficient 
but that acceptance by line management is also needed if a 
program is to be successful. 

Although Cohen and Jaffee [16] don't offer any advan- 
tages not previously mentioned, they do make three predictions 
for the future of assessment centers in government applications: 

(1) that development will become a more important objective 
than selection or internal promotion; (2) that future programs 
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will utilize a greater number of new exercises simulating 
management situations in government, and (3) that the number 
of assessment centers in government will increase, as will 
interest in them. 

Miller [50] discusses the nature of potential and 
intimates that a single job may be an inadequate measure of 
a man's full potential: 

The so-called potential of an individual is not a single 
ability, but a combination of abilities to perform effectively 
on a number of specific jobs. 

Slevin [59] writes to the same problem and points out the 

inadequacy of traditional personnel procedures: 

Thousands of psychological tests have been developed 
for the specific evaluation of personality traits and mental 
aptitudes, but they tend to be rifle shot approaches that 
evaluate some narrow aspect of executive potential. 

He goes on to point out that: 

In contrast, the assessment center provides a"broad band" 
approach to the evaluation of executive potential. 

Another point mentioned frequently in the literature 
is the effect of job assignment on an individual's career. 

Miller [50] describes it thus: 

The company is apt to place its high potential personnel 
in more challenging jobs, whereas low or no-potential per- 
sonnel will be left where they are or reassigned to less 
responsible positions. Thus they are offered little or no 
opportunity to display their qualifications for promotion. 

Thus, if such assignment is based on valid selection procedures, 

a good man may be placed in a dead-end job and left to stagnate. 

Miller refers to this hopeless situation as the "Darwinian 

theory of natural selection," and points out the hope 

offered by assessment techniques. Since all candidates compete 

on an equal basis regardless of current assignment, the "have-nots" 

are not handicapped. 
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Pomerleau [53] makes this same point, but carries it 
further. He points out the "image effect" whereby managers tend 
to reward subordinates who act like them. This can utlimately 
lead to disaster, since: 

Selecting an individual primarily because he has no 
apparent weaknesses or because his background and attitudes 
correspond to incumbent executives often leads to the 
selection of mediocrities who have no perceptible strengths. 

(For a more detailed treatment of this topic, see Senger [58]). 

Pomerleau concludes that the assessment center is the solution 

to biased supervisors' ratings since it: 

. . . ensures that the individual is observed and evaluated 

by a team of trained evaluators unbiased by previous associ- 
ations, who are familiar with the skills to be measured, 
the personal characteristics desired, and the environment 
in which the work is performed. 

A major advantage of assessment centers is the relative 
freedom from charges of bias in selection practices. Pomerleau 
[53] expresses the situation thus: 

Insofar as the assessment center technique is strictly 
job-related, employing exercises which seek to maximize 
objectivity, and further, since the concept has never been 
challenged by EEO (Equal Employment Opportunity) advocates, 
promoters of the concept may be encouraged. 

This point is amplified in an article appearing in the 
August 28, 1971 issue of Business Week : 

Dr. Cabot Jaffee thinks that, since the assessment 
method is a job sample, it may be a way to avoid charges 
of discrimination that sometimes crop up over paper and 
pencil tests. He cites a Labor Department project 
where he put a group of black and Indian trainees through 
conventional written tests and then through an assessment 
center. The conventional tests showed promotional potential 
in only 15% of them. Assessment found it in 50% of them. 
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Because of this built-in "anti-bias" factor, the assess- 
ment technique may get a boost from a recent Supreme Court 
ruling. Last March, the court ruled that, when a company 
uses personnel tests or degree requirements to screen candi- 
dates for a j ob or for a promotion, it must be able to 
show the connections between its screening method and the 
job. The court should be satisfied with assessment, since 
it's based entirely on the job. 

Jaffee, et . al . [36] address this subject in depth 

as does Dunnette [109] . A slightly different aspect of this 
general topic is covered by Bray [8] who writes on opportunities 
for women. He concludes that: "The assessment center can go 

a long way . . . toward opening up greater opportunities for 

the female employee and bringing about fuller utilization in 
the shortest possible time." 

Despite the sources previously cited, there are some 

caveats in the field of EEO which must be considered. Bray and 

Moses [76], in a review of the literature from 1968 through 

1970 offer these findings and conclusions: 

. . . considerable attention was devoted to developing 

procedures which would result in similar test performance 
among racially mixed groups. Generally, none of these 
approaches have proved to be successful. The attempts to 
assess and compensate for bias in testing procedures have 
considered the following aspects: developing "cul ture - fair" 

tests; using nonverbal tests; providing special test 
conditions; providing special test training . . . 

The culture-fair fad appears over. One factor hastening 
its demise was the finding that nonverbal tests did not 
create additional fairness for disadvantaged group members. 
Nonverbal tests were initially viewed as approximations 
of culture-fair tests. Yet one of the few findings that 
consistently emerges in several studies is that nonverbal 
tests enhance differences. 

Howard [34] provides an excellent conclusion to this 
section on assessment center advantages with the following 
summary of possible benefits: 
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Help with the Criterion Problem - - Instal lation of assess- 
ment procedures may force better job analyses and identifi- 
cation of the important criteria for success on a job. 

Such a rigorous process has been aptly described in con- 
nection with an analysis of the job of foreman* • • Another 

way assessment centers may help with the criterion problem 
is by training assessors to evaluate more accurately the 
performance, behavior, and potential of others. Assessors 
have been shown to have greater agreement in ratings of 
different assessee traits . . ., but it has not yet been 

demonstrated that assessors will experience a transfer of 
training in rating subordinates under the unstandardized 
conditions of the normal work experience. 

T raining Assessors - -Benefits of assessor training have 
been claimed not only in the form of a partial solution of 
the criterion problem but through (a) improvement in inter- 
viewing skills, (b) broadening of observation skills, 

(c) increased appreciation of group dynamics and leadership 
styles, (d) new insights into behavior, (e) strengthening 
of management skills through working with simulations, and 
(f) broadening one's repertoire of responses to problems. 

No well -designed training studies have validated these 
promises, however; as has been pointed out previously, 
firms do considerably more management research on selection 
than on training and development . . . . 

Development of Assessees - -Since many exercises, like 
the in-basket and oral presentations, were used formerly 
as training exercises, many assume they serve such purposes 
in assessment centers, even without immediate feedback of 
results .... Again, evidence supporting this training 
benefit has not been convincingly provided. Claims of 
increased self-insight . . have not been evaluated with 

pretests or control groups, and statements by assessees 
that they felt the program was informative and useful for 
self -development . . . cannot be accepted as firm empirical 

demonstrations of the developmental value of assessment 
centers . 

Minority Group Selection - -Recent government interest 
in the fairness of selection tests for minorities has 
stressed that selection procedures must be job related, 
and the simulated aspects of assessment centers do have 
face validity in this respect. One study bearing on this 
problem at AT^T . . . demonstrated that there were highly 

significant correlations between performance in a one day 
approximation of the company's longer Personnel Assessment 
Program and performance in the latter, regardless of race 
or sex. The research design in this study more closely 
resembled al ternate - f orm test reliability than predictive 
validity, however; thus the minority group fairness 
question for assessment centers is not yet sufficiently 
answered . 
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Face Validity - -The simulation exercises in particular 
have high face validity, and the whole process has been 
claimed to be received favorably by managers, especially 
those who may be mistrustful of tests .... Some assessee 
questionnaires have also indicated that the majority consider 
the procedures useful and objective .... 

Attitude Changes --It has sometimes been claimed that 
assessees may change their attitudes in the direction of a 
clearer understanding of some of the problems facing the 
managers and the necessity for making some unpopular 
decisions . . , but so far the evidence is anecdotal. 

In summary, the bonus benefits ... of assessment 
centers sound promising but are largely without research 
support . 

4 . Disadvantages (Criticisms of the concept) 

Hinrichs [32] reports a study, involving 47 Ss, where 
predictions of management potential obtained from a two day 
assessment program were compared with predictions obtained by 
a traditional technique. The traditional technique consisted 
of decisions made by two experienced managers who were provided 
with the Ss personnel records. Although the managers were 
encouraged to contact the Ss supervisors by telephone for addi- 
tional information, only a few such calls were made. Results 
of the study showed that the managers' predictions: "... 

were as highly correlated with the assessment center data as 
were overall ratings from the two-day program, except for 
ratings dealing with interpersonal behavior." The point of 
Hinrichs' study- -and a criticism of it were made by Howard [34] 

Hinrichs argued from this study that the expensive two- 
day assessment program may be unnecessary when much of the 
same information can be obtained so much easier. Inter- 
personal relationships seemed to be a relatively untapped 
area in the traditional system, however. Dunnette . . . 

[82] . . . also disputed Hinrichs' conclusions in that his 

correlation of .46 still left nearly 80 percent of the 
assessment ratings' variance unaccounted for. 
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To this writer, the real test would seem to be how 
well personnel records or performance appraisals vs. 
assessment ratings can predict future management perfor- 
mance, not how well personnel records can predict assess- 
ment ratings. 

Another critical article is that by Wilson and Tatge 
[70]. Their main thesis parallels that of Hinrichs, e.g., 
that other instruments (paper and pencil tests, clinical 
and patterned interviews, and interpretation of performance 
records) can rival assessment center results at a much lower 
cost, stating that: 

In the writers' opinion, there is' simply no available 
or published evidence to support claims of the significant 
superiority of the centers over traditional methods. 

Although the authors challenge many claims made for 
assessment centers, they fail to offer any evidence to support 
their challenges. Like Hinrichs, they conclude that the 
greatest benefit of assessment centers is for early identi- 
fication when there is no performance record to provide an 
alternative evaluation. 

Odiorne [98] comments on a study of the AT§T program 
done on a sample of 500 managers some of whom had not been 
assessed and some of whom had received various ratings at the 
assesssment center. 

This doesn't prove validity since, as Bray and Campbell 
report, the assessment center is rarely overruled, and that 
is what it proves ... A high assessment center rating 
seems to be itself a criterion for success, not a predictor 
of success. 

This was a valid criticism of the program in an opera- 
tional environment, but it completely ignores other studies 
which controlled for such contamination. Bray and Grant [10] 
describe one: 
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The Bell System's Management Progress Study (Bray, 1964) 
offers a unique opportunity to study the assessment process 
. . . The uniqueness of the study from the standpoint of 

studying the assessment process arises from several aspects 
of the study design: 

1. There is no contamination by the assessment results 
of the subsequent criterion data. Along with all other infor- 
mation collected on the 422 subjects of the study, the 
assessment data are being held in strict confidence. Thus 
the judgements of the assessment staff have had no influence 
on the careers of men being studied. 

The other assessment problem cited by Odiorne was 
the effect of a rating of failure on an employee's later perfor- 
mance. This is spoken to by Kraut and Scott [41] . They found 
that the lowest rated group of candidates had a mobility rate 
one-half to two-thirds that of the total and therefore concluded 
that a poor assessment will slow promotion but not necessarily 
deny it. 



As with the previous section, Howard [34] provides a 
succinct and comprehensive summary of the field as follows: 

The Crown Prince or Princess . Those who do outstandingly 
well in assessment centers may find that they have become a 
crown prince or princess. Management may treat them so well 
that their future success becomes a self-fulfilling prophecy, 
the morale of those without royal status may decline, and 
the validity coefficients for the assessment center process 
may become inflated. No research has substantiated these 
potential coronation effects, however. 

The "Kiss of Death" . A candidate who does poorly at 
an assessment center may feel that he has been given the 
kiss of death as far as his future with the company is con- 
cerned. This could result in some undesirable attrition, 
since the candidate may be quite competent in the job he is 
now performing. Research on turnover of assessees so far 
has been inconclusive. 

Stress. If a candidate gets the impression that his 
entire career is on the line based on a few days "on stage,'.' 
the stress effects could be quite strong. It would seem 
important that the data from the procedure not be made of 
the pass-fail variety or kept too long in an employee's 
file. On the other hand, defenders of the procedures reply 
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that since stress is a typical part of a manager's job, 
a candidate should be stressed to see how he copes with 
it. It would still seem important to keep stress in the 
exercises with limits. 

The Nonnominee . The feeling that an individual may 
be part of the "out group" if he or she has not been selected 
to participate in the assessment process (whicli may become 
a status symbol) is another dimension of employee attitudes 
that needs to be empirically tested. 

The "Organization Man. " Some have raised the issue of 
whether or not assessment centers may not proliferate the 
model of the conforming organization man and serve to 
eliminate the unusual or imaginative managers that are 
believed to be needed in the future. A study at Sohio 
showed that assessments correlated negatively with con- 
formity for one small sample, however .... An IBM study 
indicated that supervisors may nominate those higher on 
conformity and lower on independence, but that the assess- 
ment procedure itself does not select this type of individual 
. . . . The organization man may be the other side of the 

nonnominee problem; the most able and not the least able 
may be denied access to the assessment center. The impli- 
cation is that it is the nomination procedure and not the 
assessment procedure that creates the organization man 
syndrome. The supervisory nominations should perhaps be 
supplemented by self -nominations , peer nominations, personnel 
records, or assessment of everyone at a job level if numbers 
are not too large. 

Costs . Estimates of costs have ranged from the price 
of a few meals to $5,000 per candidate, exclusive of staff 
salary .... Installation costs are the highest, but to 
these must be added assessors', assessees' , and psychol- 
ogists' time, travel, accommodations, and meals, plus 
materials, from rating sheets to videotapes. Various cost 
saving devices might include completing all possible pro- 
cedures before arrival at the center, conducting exercises 
on company property over weekends, and combining small 
companies with similar jobs in a multiple company center, 
perhaps in a synthetic validity paradigm. In the end, 
these costs must be weighed in the context of current 
selection ratios against the possible gains in selection 
and training in some kind of a utility model. 

In summary, the possible negative outcomes of assess- 
ment centers, have much the same status as the (benefits), 
they appear reasonable, but for the most part they lack 
supporting data. 
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5 . Design Considerations 

This section will review those factors seen as relevant 
to the design of an assessment center. In this area, Finkle 
and Jones [85], the Foundation for Research on Human Behavior 
[112] and Olmstead, et . al . [122] are the most pertinent and 

comprehensive references. 

a. Dimensions to be Assessed 

The three sources just cited cover this topic quite 
thoroughly. Miller [50] and Kraut [40] offer general comments 
on how to ascertain relevant job dimensions. Kraut offers some 
particularly cogent advice: 

Most assessment programs are not based on an empirical study 
of the manager's role, as might be done through careful job 
analysis or a critical incident study. At best, they are 
based largely on a review of the research literature and 
the judgment of executives in the organization as to what 
makes for an effective manager. At w^orst, they tend to 
be copies of programs in other companies. 

Helrne et . al . [114] describe the development of 

a combat simulation, and Krumm et. al . [118] writes on a 

research project to explore tactical military decision making. 
Although these articles are quite narrow in scope, they both 
deal with designs of military simulations and should be of 
some value to an assessment program designer. 

b. Number and Character of Assessments 

Several sources in the field suggest that more 
than one assessment of an individual is needed during a career. 
The basis underlying such a concept is outlined in Hall's paper 
[30] "A Theoretical Model of Career Subidentity Development 
in Organizational Settings." Hall. defines career subidentity 
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■as : 



. . the person's conception of himself in his career 

He goes on to explain that subidentity (occupational) 



1 1 



role . " 

selection "... is not simply a matter of selecting a role 

but also one of choosing an aspect of one's self which is 

potentially effective and highly valued." He underscores the 

importance of this decision by pointing out that: 

. . . the career subidentity grows at an enormously faster 

rate in the first year than in subsequent years . . . the 

early years undoubted] y represent a critical period for 
learning and leave a lasting imprint on the person's atti- 
tudes and aspirations. The expectations and standards 
internalized at this time will probably be enduring. 

These comments indicate a need for accurate assess- 



ment at the point of occupational selection (career entry) . 



Hall then introduces the concept of career develop- 



ment which he defines as a " . . . spiraling combination of 

career choice, subidentity growth, and commitment." He 



emphasizes the difference between occupational choice and 

career choice, explaining it as follows: 

Occupational choice, the choice of a career role, is made 
but two or three times in most lifetimes, while career 
choice, any choice which will affect one's career develop- 
ment, can occur continually. 



He points out that: 

. . . this focus on postselection career development repre- 

sents a departure from the usual psychological concern for 
occupational choice--the process of choosing a career role 
congruent with characteristics of the individual. 



These latter comments indicate that the need for assessment 



does not end once the occupational choice is made, but continues 
to exist as a result of the career choices which "can occur 



continually. " 
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Pomerleau [53] expresses the same basic concepts 
in a more pragmatic way: 

When we speak of an "identification system," we need to 
be reminded that it has two subsystems: (1) Parly identi- 

fication of high potential (EIHP) and (2) Identification 
of higher potential within the midmanager ranks. Funda- 
mentally, the first subsystem asks: What could the raw 

candidate, devoid of any meaningful performance record 
beyond academic achievements, do, given the opportunity to 
perform in challenging tasks? The second subsystem asks: 
What has the midmanager done so far to distinguish himself? 
Our source of data can be culled from job performance, 
•’experience, developmental assignments, supervisor appraisal, 
etc., and then determine what kinds of plans could be 
mapped out for him to realize the full measure of his 
managerial potential. 

Slevin [59] has a slightly different view of the 
concept and is the first to specifically describe the "assess- 
ment center approach:" 

The assessment center is conducted in the usual fashion 
with an emphasis on botli evaluation and the diagnosis of 
developmental needs. These two variables may then be fed 
into an overall manpower plan. This plan takes into account 
the environmental demands on the firm and the projections 
of future manpower heeds in various areas. Based on his 
own competence and what his developmental needs are, an 
individual will be plugged into two plans: a promotion 

program that is specifically constructed for the individual, 
and a management development program. The promotion pro- 
gram specifies a career path that the individual is likely 
to follow in the organization. The management development 
program focuses on the difference between the individual's 
current competence and future needs as defined by his 
tentative career path. Using both the promotion and 
management development programs, the organization can 
implement an effective and farsighted manpower plan. 

The assessment center is probably the best example of 
a successful "broad band" approach. Because it is able to 
fill both organizational appraisal and developmental needs, 
the assessment center is likely to flourish as we prepare 
our organizations for the 1980' s. 

Miller [50] makes one of the most explicit state- 
ments on the subject: 
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Moreover it is an error to assume that an individual's 
capacity for promotion to the next higher level or to jobs 
several levels higher remains constant. Job requirements 
change, organizational goals and demands shift , and an 
individual's personal qualifications are likely to change 
with time; hence his potential for promotion to various 
jobs will vary in time. Therefore potential must be period- 
ically reassessed. 

In conclusion, Hinrichs [32] summarizes the diverse 

objectives of an assessment program as follows: 

. . . one hopes the assessment program will be able to 

identify promotable . people earlier in their careers, that • 
it will help to clarify some of the skills important in 
promotion, and that it will perhaps identify some people 
who should be promoted but who might, under normal circum- 
stances, be overlooked. Hopefully, also, the program will 
fulfill, to some extent, a personal development function 
by providing practice in group situations, individualized 
feedback regarding observed strengths and weaknesses, and 
greater understanding regarding the caliber of the competi- 
tion participants are up against. 

c. Uses of Results 

The output or result of the assessment process can 
be utilized in several different ways. This is not to imply 
that a single program could meet all of the diverse objectives, 
for some of the objectives are in conflict, e.g., selection 
usually calls for stable variables to be measured, while 
unstable variables are the focus of developmental programs. 
Because of this, it is important that objectives (primary and 
secondary) be specifically delineated before the program is 
designed, to ensure highest efficiency. 

Tinkle and Jones [85] cover this topic most 
thoroughly in Chapter 15 of their book. They identify the 
following uses: 

Selection decisions 
Hiring 
Firing 
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Transfer and placement 

Organization requirements 
Individual requirements 

Management development 

Organizational planning 

Counseling and consulting 

Additional uses suggested by Culclasure [108], are 

career motivational prediction and selection. Although his 

study failed to develop predictors, lie wrote: 

Available evidence, however, suggested that a career moti- 
vation screening procedure could be developed . . . Such 

a procedure would employ (a) those portions of the Strong 
Vocational Interest Blank (SVIB) which have been shown to 
have moderate validity (0.25) for predicting Naval officer 
retention when scored with the Navy Officer Key, and (b) 
a revised version of the Importance-Possibility Scale. 

Helme, et. al. [114] also address military assess- 



ment applications as follows: 

Early identification of officer leaders and development 
of officer leadership from cadet training through company 
and field grade assignments are of major concern in the 
management of the Army's manpower resources. The Behavior 
and Systems Research Laboratory (BESRL) conducts research 
to provide scientific means of identifying individuals with 
good leadership potential for officer training, selecting 
officers for commissioning, and evaluating their performance. 
The present series of publications records the conduct of 
a long-term experimental program to improve the basis for 
selecting and developing officer leaders in accordance with 
their capability to meet differing leadership requirements. 
Differential prediction and evaluation have become dominant 
objectives in the effort to channel officers into appropri- 
ate assignments and develop their potential so as to make 
best use of their abilities. 

As mentioned previously, a single assessment 
program is realistically confined to one or two objectives. 

This is illustrated by the fact that studies of operational 
assessment centers rarely address more than two objectives. 

The following examples are offered. 
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Miller [50] cites selection and identification; 
Pomerleau [53] stresses identification with a secondary function 
of development; Slevin [59] says essentially the same thing, 
stating: "No two problems are more crucial in today's changing 

organizations than effective management appraisal and develop- 
ment." Kraut and Scott [41] describe a combination of assess- 
ment and development: 

The assessment program consists of two and one half days of 
assessment activity patterned after the AT§T model, plus two 
days of purely developmental activity . . . The development 

portion includes lectures, seminars, and classroom discussions, 
as well as a feedback interview. 

Jelks [37] makes an important point regarding the 

use of assessment center information: 

The program is not designed as a final judgment factor in 
the selection of management personnel, nor does it replace 
the supervisory appraisal of current job performance. It 
simply makes one more piece of information available that 
can be fitted into a record of employee performance. 



Kraut [40] makes the point just stated, and another, 
equally important one: "The data should ... be discarded for 

any decision making purpose after two years or so." Interviews 
disclosed a similar policy at Pacific Bell Telephone with a 
figure of about three years mentioned as a maximum "life" of 
data for selection purposes. In contrast, Finkle and Jones [85] 
cite a period of "five to ten years, or longer." 

The final source in this section makes a prediction 

on the future uses of assessment centers. Bray and Moses write: 

Byham's . . . [11] . . . review of personnel research activ- 

ities indicates a growing interest in developmental and 
placement strategies rather than on selection per se. 
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After deciding on the objective(s) of an assess- 
ment program, the designer next faces a myriad of decisions. 

The more basic of these will be arbitrarily lumped together 
and considered in the next section, 
d. Basic Considerations 

Three factors considered basic to assessment program 
design will be reviewed. These factors are: (1) "Signs vs. 

samples," (2) Stable vs. unstable variables; and (3) Statistical 
vs. clinical combination of assessment results. 

(1) Signs vs. Samples . Bray and Moses [76] write: 

Wernimont and Campbell . . . [67] . . . argue that tests and 

other predictors should be used as samples rather than signs 
of behavior. In other words, the measures we use to predict 
should be measures of behavior. 



Howard [34] seconds this view by noting that: 

A unique contribution of assessment centers is the inclusion 
of situational tests in the assessment battery. The rationale 
behind using such exercises is that they simulate the type 
of work to which the candidate will be exposed and allow his 
performance to be observed under somewhat realistic conditions. 
Contrary to the aptitude test approach, samples, not signs 
of behavior are used for prediction. 

Wernimont and Campbell [67] contend that samples are superior 

to signs in the following ways: (a) less bias due to faking 

and response sets; (b) less discrimination in testing because 

samples are more relevant and fair than signs; (c) less invasion 

of privacy with samples. As to the relative effectiveness of 

"signs vs. samples," Bray and Grant [10] stated that: 

The data reported make it apparent that the situational 
techniques (group exercises and .In-Basket) used in the 
Management Progress Study produced, despite their complex- 
ities, reasonably reliable results and that they markedly 
influenced the judgments of the assessment staffs. The 
paper-and-pencil instruments (signs) had less influence 
on staff evaluations generally, though they did influence 
them in many specific ways. 
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Wollowick and McNamara 1.71] refer to the preceding source and 

speculate as to the cost-effectiveness of simulations (samples): 

An important question frequently raised (Bray and Grant, 

1966) can also be partially answered from these (Wollowick 
and McNamara's) results. That is, can the assessment pro- 
cedure be justified in light of its additional cost and 
time compared to the use of paper-and-pencil tests alone? 

. . . Inclusion of the elements unique to the assessment 

center procedure . . . nearly doubles the criterion variance 

accounted for. This indicates that the assessment procedure 
makes a substantial unique contribution to the prediction 
of management success. 

(2) Stable vs. Unstable Variables . The considera- 
tion given to this question in the design of the Sohio program 
is described by Finkle and Jones [85] : 

. . . the emphasis in the program at Sohio was on the 

development of information that might be expected to be 
reasonably stable and consistent over some period of time, 
perhaps five to ten years or longer. Studies have been 
made as to the degree to which measurements and judgments 
of abilities, interests, attitudes, and values of individuals 
tend to change over time. It seems appropriate, in establishing 
a group of basic abilities and characteristics, to make use 
of available studies, general experience, and common sense 
to eliminate from consideration variables that are highly 
vulnerable to change during a relatively short period of 
time . 

However, this approach clearly presupposes that the 
program will establish less information pertinent to training 
and developmental plans than might otherwise be possible. 
Immediately the question arises: Is it not possible to do 

both, namely evaluate some characteristics not likely to 
change and others more likely to? The answer, of course, 
is "yes." Both approaches could be taken, but the orienta- 
tions are essentially in conflict. Our emphasis in Sohio 
was placed on stability. It was not planned to reschedule 
the elaborate assessment procedures of this sort in any 
short period of time since it was felt that it was important 
to be as fully aware as possible of more permanent charac- 
teristics when making manpower decisions, particularly those 
of a long-range nature. 

(3) Statistical vs. Clinical Combination of Assess - 
ment Results . Bray and Grant [10], in describing the ATfjT 
Management Progress Study assessment program, make a rather 
one-sided statement: 
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Assessment procedures also contrast with psychometric 
ones in the way the resulting data are combined. Psychometric 
approaches depend on mathematical methods for accomplishing 
this purpose whereas assessment approaches combine the data 
judgmentally . 

Taft [61] is more objective on this point and offers an excellent 
analysis of both sides of the question: 

We have argued that there are occasions when intuitive 
methods of making predictions, i.e., "clinical" have their 
appropriate place. Statistical methods cannot be used 
where no prediction formula exists. But some personality 
assessors speak as if the clinical method is always to be 
preferred as it enables the assessor to be flexible in his 
use of the data in a way that is not possible with statis- 
tical techniques; for example, the clinician can give weight 
to obvious but rare and nonrepeatable factors in the subject's 
current situation which could not be validated empirically. 
Other advantages claimed for the clinical against the 
statistical approach are that it does not violate the essential 
unity of the subject's personality, and that it enables the 
use of empathy and recipathy in making the predictions. 
(Actually these subjective clues could also be used as data 
by the statistician along with other more objective data.) 

Other assessors regard clinical techniques as only a 
last resort. A number of advantages can be quoted for 
statistical prediction over clinical, most of which boil 
down to the fact that the statistician has a far more effic- 
ient memory and a larger attention span than the clinician; 
he can "remember" the relevant data at the appropriate time 
and combine them with other data in order to obtain optimal 
weightings for future predications. 

And so we have, on the one hand, the efficient but rigid 
and inhuman statistical prediction, and on the other, the 
flexible and humane but inefficient clinical. Which one is 
more useful in personality assessment? 



The choice of method will depend on both the requirements 
and the over-all situation, including sometimes, public 
relations considerations. The final selection of assessment 
techniques is likely to be a mixture of both subjective 
and objective, but the circumstances that will favor one or 
the other at any stage are rather vague, and the choice is 
usually made on subjective grounds, although it, too, could 
be made on the basis of objective, empirical investigation. 

In general, objective methods are to be preferred as far as 
possible as they maximize accuracy, but practical consider- 
ations of economy, convenience, and the limitations of the 
situation, dictate the wholesale use of subjective methods 
in personality assessment. These subjective methods may 
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have high validity under favorable circumstances, and where 
the assessors are familiar with the criterion situation, 
clinical judgments may actually be more accurate than any 
objective methods are ever likely to be in predicting to 
criteria . 

Wollowick and McNamara [71] report an interesting finding from 
their study: 

The subjectively derived combination of the variables (overall 
rating) correlated .37 with the criterion, while the statis- 
tical combination gave a multiple of .62. This suggests that 
instead of deriving an (overall rating) by subjective mean(s) , 
it might be done more profitably by a statistical procedure. 
This should greatly increase the predictiveness of the 
program . 



Meehl [96] makes a very positive statement on 
the issue as he summarizes a number of studies on this subject: 

In spite of the defects and ambiguities present, let me 
emphasize the brute fact that we have here, depending upon 
one’s standards for admission as relevant, from 16 to 20 
studies involving a comparison of clinical and actuarial 
methods, in a!3 but one of which the predictions made 
actuarially were either approximately equal or superior to 
those made by a clinician . (italics are Meehl' s.) Pur the r 
investigation is in order to eliminate the defects mentioned, 
and to establish the classes of situations in which eacli 
method is more efficient. 

Lest Meehl' s position be misconstrued by quoting him out of 
context, his "thesis in a nutshell" is quoted: 

There is not convincing reason to assume that explicitly 
formalized mathematical rules and the clinician's creativity 
are equally suited for any given kind of task, or that their 
comparative effectiveness is the same for different tasks. 
Current clinical practice should be nuch more critically 
examined witli this in mind than it has been. 

One final viewpoint on this subject is offered. 
Howard [34] cites yet another study, and makes some suggestions 
for future assessment programs: 

Again we find a demonstration of the usefulness of 
clinical measurement but the superiority of mechanical 
combination of data . . . (Sawyer [56]). Although one 

study cannot be generalized to all assessment centers, 
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it certainly appears advisable for other centers to research 
the hypothesis that mechanical combination of data may improve 
predictions even more. Should this prove true, once research 
costs were recovered, the unit cost savings of reducing 
assessors' time could be substantial. 



e . Ins 
The 

formidable task 
which will best 
liis task is the 
approval . 

Alb 



truments and Techniques 
designer of an assessment program faces a 
in the selection of instruments and techniques 
serve the objectives of the program. Compoundin 
plethora of commercial instruments vying for his 

right et . al . [73] offer a relatively brief but 
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comprehensive coverage of "the use of psychological tests in 
industry." Bray and Grant [10] provide an overview more relevant 
to assessment programs with their listing of the instruments 
included in the Management Progress Study: 

The methods used for collecting information on the 
personal characteristics of the participants are representa- 
tive of those used generally in assessment activities. A 
listing of the techniques with a brief description of each 
follows : 

Interview . A two-hour interview with each man directed 
at obtaining insights into his personal development up to 
.that time, work objectives, attitudes toward the Bell System, 
social values, scope of interests, interpersonal relation- 
ships, idiosyncrasies, etc.. 

In-Basket . A set of materials which a telephone company 
manager might expect to find in his in-basket. The items, 

25 altogether, range from telephone messages to detailed 
reports. In addition, examinee was furnished with such 
necessary materials as a copy of the union contract, organi- 
zation chart, and stationery. He was given three hours in 
which to review the materials and take appropriate action 
on each item (by writing letters, memos, and notes to himself). 
Following completion of the "basket" he was interviewed con- 
cerning his approach to the task, his reasons for taking 
the actions indicated, and his views of his superiors, 
peers, and subordinates (as inferred from the materials). 
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Manufacturing Problem (made available by John Hemphill 
of the Educational Testing Service) . A small -bus iness 
game wherein the participants assumed the roles of partners 
in an enterprise manufacturing toys for the Christmas trade. 
The participants were required to buy parts and sell finished 
products under varying market conditions, to maintain inven- 
tories, and to manufacture the toys. 

Group Discussion . Also a leaderless group situation, 
focused around a management personnel function. Participants 
were instructed to assume the roles of managers, each having 
a foreman reporting to him considered capable of promotion. 
Participants were required to discuss the merits and 
liabilities of their hypothetical foremen and to reach a group 
decision regarding their relative promotabil ities . 

Pro j ectives . (a) Rotter Incomplete Sentences Blank 
(published by the Psychological Corporation) . (b) Bell 

Incomplete Sentences Test (by Walter Katkovsky and Vaughn 
Crandall with the advice and assistance of Julian Rotter) . 

(c) Thematic Apperception Test (published by Harvard 
University Press). Six of the cards from this test were 
administered. 

Paper-and-pencil Tests and Questionnaires . (a) School 
and College Ability test, form 1 (published by the Coopera- 
tive Test Division of the Educational Testing Service) . 
Annually since 1956 the Personnel Research Section of ATqT 
has developed, following the Educational Testing Service 
format, its own version of this test. (d) Edwards Personal 
Preference Schedule (published by the Psychological Corpor- 
ation) . (c) The Guilford-Martin Inventory of Factors GAMIN 

(published by the Sheridan Supply Company) . (f) Opinion 

Questionnaire, Form B. Unpublished, this questionnaire, 
made available to the Bell System by Irving Sarnoff of New 
York University, is designed to reflect a person's attitudes 
toward making money and advancing himself. 

Miscellaneous . (a) Personal history questionnaire. 

(b) Short autobiographical essay. (c) Q sort (70 items, 
self-descriptive) . 

Another overview can be found in Finkle and Jones 
[85]. They provide the following informative table: 
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Figure 7 

Suggested Factors Indicating the Value of a Test or Procedure 



IMPORTANCE 



HIGHER THE BETTER 



LOWER THE BETTER 



Of critical 
importance 



Relevance 

Validity (professional 



Unacceptabil i ty 
to assessees 



confidence) 



Of considerable 
importance 



Uniqueness 
Acceptability to 



Time to administer 
and process 



management 
Acceptability to 



assessees 



Cost in time of 
management 
and professional 
personnel 



Cost to purchase 



Cost to develop 
or improve 



Of some importance Change of pace 



Restriction on 
research 



Source: Finkle and Jones [85] 



They continue with some additional suggestions to consider in 
the selection or design of tests: 

Mental Ability Tests 

Beyond the general criteria of relevance, uniqueness, 
acceptability, and the others already considered, it is 
helpful in choosing mental ability tests for the assess- 
ment program to seek ones for which appropriate norms are 
available. Here we mean tables or statistics showing what 
would be considered typical, or relatively high or low, 
scores on the test for persons similar, in whatever quality 
the test measures, to those to be assessed. Preferably, 
tests can be used for which norms on managers or executives 
in business are available. Such information can usually 
be obtained through test publishers or often by cooperative 
arrangement with business organizations that have vised the 
test. Later, as sufficient experience is gained, the com- 
pany with the new assessment program can establish its own 
norms and standards based on scores of its own personnel. 

Projective Personality Tests 

In projective tests the psychologist is considered to 
be part of the test. Therefore, the choice and make-up 
of projective tests should, to a considerable extent, be 
left in the hands of the clinical psychologist who uses 
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them. However, it may be well, in choosing the clinical 
psychologist who will particpate in the program, to consider 
whether the projective tests he prefers to use will prove 
generally acceptable to assessees and managers in the organi- 
zation and will be acceptable to other clinical psychologists 
who may help him or replace him in the program. We doubt, 
for example, that ink-blot or word-association tests are as 
acceptable in business as thematic or sentence-completion 
tests. And if the sentence -comple tion procedure is to be 
used (particularly if one is to be tailor-made for the 
program) , it becomes important to have some assurance that 
it is acceptable to psychologists other than the one who 
designed it. 

Multiple- Choice Personality and Interest Tests 

This form of testing has been the principal focus of 
negative criticism of management level testing in business. 
Accusations have ranged from unfairness, through perpetuation 
of conformity, to simple ridiculousness. Anyone considering 
the use of such procedures should obtain good professional 
counsel since much of the criticism, though so frequently 
and sensationally expressed as to alienate most professionals, 
has some basis in fact. 

Two characteristics of such tests probably account for 
most of the criticism leveled at them. Tirst, they rather 
obviously can be faked. This by no means necessarily 
results in a distortion that is favorable to the "faker." 

It does, however, reduce his confidence in the interpretation 
of the test and therefore in the fairness and value of the 
assessment process. Furthermore, it may prevent the effec- 
tive use of the scoring and weighting system originally 
developed for the test without, at least, a follow-up study 
to examine the effects of attempted faking on the scoring 
system that was, in most instances, developed from answers 
given by persons taking the test experimentally or for 
counseling purposes. Second, the basis of development and 
scoring of such tests is actuarial rather than clinical. 

This means that the choice of items and the scoring system 
are ultimately based on statistical relationships between 
actual answers to the questions and one or more other 
measures such as judgments of performance or personality 
constructs. This relationship is accepted as an empirical 
fact and efforts at making a rational interpretation of 
individual item relationships are usually not attempted 
or considered relevant. Consequently, anyone judging the 
contribution of the test by reading and thinking through 
the answers, or by examining the weights given the answers, 
may simply miss the mark in evaluating the test. 
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These characteristics of multiple -choice personality 
tests point to the importance of a careful examination of 
their value under conditions of actual use as a means of 
weighting their contribution against their dubious accept- 
ability. In selecting tests a review of the questions for 
study can give some clue as to the reaction that can be 
anticipated by test-takers. 

Interviewing 

Interviewing can presumably be called a type of assess- 
ment procedure, but there seems to be nearly as many ways 
of interviewing as there are interviewers. As discussed 
earlier, arguments can be made for having all interviews 
in an assessment program conducted by psychologists or 
other specially trained persons. Other arguments support 
the value of having managers conduct the interviews within 
the frameworks of some assessment programs. The Sohio 
program adopts the latter approach and in fact calls for 
a constantly changing (from program to program) group of 
managers to conduct interviews with a minimum of special 
instruction or training. 

Clearly, the value of interviewing under such conditions 
comes from maximizing the already developed habits of the 
managers rather than in attempting to change or "perfect" 
their skills in accordance with some predetermined style or 
pattern of interviewing. In a brief "orientation" meeting 
prior to their conducting interviews, the managers are 
asked to review personal history forms of the four individuals 
each of them will interview. It is pointed out that these 
interviews differ from ones they have held in the past in 
that this time 'they have no job to describe or "sell." They 
are encouraged to look for answers in the personal history 
form that can be used to encourage the interviewee to 
elaborate on why he did a certain thing such as choose a 
college, a major in college, a company to work for, or a 
change of company. .They are also asked to point out to 
the interviewee their unique role as his representative to 
the assessment committee: How would the interviewee like 

to be represented? What does he feel lie is best suited 
to do in the company and why does he feel lie is so suited? 

The managers are cautioned that some interviewees may 
attempt to turn questions back on them and it is suggested 
that they refrain from describing their own backgrounds 
and activities until the formal interview it over. 

A few sample interview reports from past programs (edited) 
are presented and it is stressed that their own reports should 
contain two things: information about the interviewee and 

their judgment about him from the interview exposure. No 
pattern is recommended in organizing the report. In fact, 
one sample interview report is written chronologically from 
childhood while. the other begins with the first Sohio job 
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and uses earlier experience information to support and 
elaborate on opinions and judgments about professed skills 
and interests. The managers are offered tape recorders if 
they do not wish to write out their interview reports in 
longhand. This approach, though it is only slightly struc- 
tured and offers little or no advice on style, has resulted 
in most instances in the production of effective 1-1/2 to 
2-1/2 page typewritten reports. 

Situational Exercises 

So far as we know, situational exercises have to be 
developed or borrowed rather than ordered as published 
versions through a catalogue. Such procedures can provide 
either individual or group activity. The overriding objec- 
tive of the approach is to produce activity based on business- 
like matters which can be reliably judged by observers or 
reviewers. It is presumed, though sometimes given secondary 
attention, that the judgments (or observations) produced 
can be weighed compatibly with other information to produce 
meaningful combined judgments helpful in future selection, 
placement, or developmental decisions. A critical aspect 
of any situational exercise, therefore, is the realism (in 
regard to business problems) in the eyes of those partici- 
pating in it and in the eyes of those judging it. Accordingly 
with either individual or group exercises, the task provided 
should bring the assessee as close as possible to realistic 
problem-solving and should require as little ,: role-piaying" 
as possible. Observers should be able to hear or observe 
enough to give meaningful interpretation to what occurs 
rather than to have to infer what a person is thinking or 
feeling. Some thought should probably be given to estab- 
lishing a variety of tasks so that persons with varied 
backgrounds, skills, and characteristics find opportunity 
to demonstrate their talents and styles. For example, 
while one group exercise might call for quick and con- 
vincing argumentation based on little specific content, 
another might call for some quantitative analysis as a 
prerequisite to discussion, while still another might 
feature sensitivity to the feelings of others as a basis 
for appropriate contribution. The exercises might also 
vary in the amount of preparation allowed in order to 
contrast planned and rehearsed behavior .with impromptu 
behavior . 

In all exercises, the determination and explanation of 
exactly what the observer is to note and report is quite 
important. In individual exercises, the observer may examine 
the results of the exercise (as in the in-basket exercise); 
he may observe individual performance (as in the presentation 
of formal talks); or he may actually question the assessee 
as to why he took certain actions (again, as in the in-basket 
exercise). In some individual exercises, observations may 
not be made, but- rather scores (or counts) may be obtained 
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by formula and reported to the assessment committee (as 
in a stock market exercise developed and experimented 
with at Sohio) . 

For group exercises, observers may have specific 
assignments to concentrate on the performance of one or two 
of a group of individuals who are engaged in a leaderless 
group discussion (such as reviewing a case report) or in 
a simulated work activity (such as a marketing -manufacturing 
problem) . Here the observer may be given considerable or 
little structure ranging from using a checklist to mark 
what he sees to simply writing notes about what he sees. 

Such notes can then provide him with the basis for preparing 
a written report or for "telling" the assessment committee 
what "his" assessees did during the exercise. 

An observer could, of course, be asked simply to com- 
plete a series of rating scales similar to or even identical 
to the skill and style variables to be used by the assessment 
committee. However, we feel that this approacli has two 
disadvantages. First, the other members of the assessment 
committee, who concentrated on other assessees or who did 
not observe the exercise at all, will have no way of joining 
in the interpretation and judgment of the performance of 
the individual being reported on; they must simply accept, 
or not accept, -the judgments of the reporting observer. 
Second, the observer may, by rating the assessee on the 
variables solely from observations in one exercise, over- 
commit himself on each rating prior to hearing evidence 
from the other exercises and procedures. To offset these 
possible problems, we suggest that observers be given a 
checklist or structure that produces broad coverage and 
descriptions without calling for judgments on the basic 
program variables. A highly detailed checklist seems 
advisable only when the observers can be we] 1 trained in 
its use - -generally when management observers are expected 
to remain as staff contributors through several programs. 

Though somewhat less important than other considerations , 
some method of ready identification by observers of all 
participants at a group exercise fosters better association 
of remarks with those making them. We have adopted the use 
of colored vests --other approaches could probably serve as 
well. Observers otherwise will have no trouble identifying 
the persons they are assigned to concentrate on- -but they 
might have difficulty in identifying exactly who their 
assigned assessees arc talking to or with in the group- - 
particularly if a couple of the "other" assessees have their 
backs to the observer. By having each participant in the 
group exercise clearly identified (from all directions) by 
color, the observers can merely note interactions by these 
symbols, and when their reports are later typed, the typists 
can substitute the correct names for the colors. 
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One final bit of advice for the designer is offered 
by Hardesty and Jones [31], e.g.: 

In addition to the variety of assessment approaches used 
in the program, the designers purposely attempted to build- 
in duplication when possible and to strive for some overlap 
among and within the assessment approaches in order to 
increase the confidence they could have in the assessment 
information . 

The remainder of this section will continue to 
.consider specific instruments and techniques individually. 

(1) Projective Personality Tests . The best 

article found on this subject is "Contributions of projective 
techniques to assessment of management potential," by Grant et , 
al . [27]. The authors conclude that "... projective reports 

particularly influenced the assessment staff in rating such 
characteristics as work motivation, passivity, and dependency. 

(2) Interviews . A number of pertinent articles 

have been written on interviews. Grant and Bray's [26] "Con- 
tributions of the interview to assessment of management potential," 
is one of them. They found that: "Analyses of the data 

clearly indicate that information from the interview reports 
contributes to assessment center evaluations." 

"The validity of personnel interview" by 
Ghiselli [23] is a brief but worthwhile article. His conclusion 
states that: • • • "the ordinary personnel interview is not 

necessarily and invariably invalid, but rather that its validity 
may be at least equal to, if not greater than, the validity 
of tests." 

Prien [54] reports a study to determine the 
effectiveness of interview information as a predictor of future 
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•job performance. lie concluded that: 

Psychologists differentiate individuals through interviews 
on two general dimensions: Personal Relations Effectiveness 

and Intellectual Functioning . . . 

and 

Psychologists can predict the potential job performance of 
individuals on the basis of information and impressions 
obtained in an interview. The interview has some validity 
for the assessment of higher- level personnel. 

(3) Simulations/Situational Tests . This category 
can be subdivided into. group and individual tests. Those articles 
dealing with the entire field will be reviewed first, followed 
by those on group exercises. Individual tests will be covered 
last. 

Finkle and Jones [85] again offer an excellent 
treatment which is enhanced by the inclusion of advantages 
and disadvantages of situational techniques: 

This approach consists of one or more standard exercises 
expected to bring out ability and personality characteristics 
directly and more obviously related to successful performance 
in supervisory, administrative, and managerial responsibil- 
ities. These exercises may consist of individual tasks, such 
as writing recommendations about a variety of letter s, reports , 
notices, and so on, typical of the contents of an executive's 
"in" basket. Or they may require elaborate group activity 
of six or eight candidates "running" a simulated company 
or "serving" on a committee. To some degree how well the 
situations are handled is informative: for example, the best 

recommendations on the in-basket items, higher profits in 
running the company, or controlling the committee. There is, 
however, some question as to whether or not results on these 
situational exercises guarantee results in a real life situ- 
ation. Much more important is the picture obtained of the 
abilities, skills, styles, and personalities of the partici- 
pants. Such characteristics are noticed as: (1) degree of 

organization, (2) willingness or reluctance to make decisions, 
(3) willingness to take action, (4) clarity and impact in 
communications, (5) awareness and sensitivity to the thoughts 
and feelings of others (6) impact on group decision-making 
and (7) approach to problem definition and attack. 



81 



The interpretation of these situational activities may 
vary considerably from a rather intuitive, commonsense, or 
clinical judgment formed by either psychologists or manage- 
ment personnel as to the significance of certain answers or 
actions to, on the the other hand, a rather empirically 
related analysis of the answers and actions as compared 
with other judgments in terms of ratings, later actions or 
future consequences. In other words, these techniques may 
be approached on the basis of the standardized selection 
techniques, such as regular paper-and-pencil tests, supposedly 
supported by validation studies against other measures 
obtained after employment; or they may be judged more as 
the clinical information is judged by the clinical psycholo- 
gist. 



Advantages 

a. The problems and situations used with this approach are 
designed to bring out characteristics and abilities that 
seem directly related and relevant to success in the business 
world. 

b. These techniques involve some important areas, such as 
interpersonal relations, administrative skills, and communi- 
cations ability that are not well covered by the other 
techniques . 

c. These techniques are readily adaptable to administration 
and use by business personnel as well as by psychologists, 

d. The descriptions of the styles and abilities of indi- 
viduals as assessed by these techniques are more readily 
understandable to management personnel making promotion 
decisions than are some of the descriptions written by 
psychologists following the clinical approach or scores 
derived from the statistical approach. 



Disadvantages 



a. This approach can be relatively time-consuming, depending 
upon the number of exercises employed. Most exercises take 
from one to three hours. 

b. There is always the possibility in the group exercises 
that one individual will look relatively strong or weak in 
some respects because of the makeup of the particular group 
in which he is placed. This and the possibility that 
observers may not be consistent in how they judge and report 
what they see make the process one of low reliability (con- 
sistency) . 

c. There is no certainty that performance under testing 
conditions will transfer to later performance under real 
conditions, particularly if results on the "correctness" 
of the answers are considered rather than style. 
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d. The cost of this technique varies according to the method 
that has been used for the interpretations of the results. 

If the empirical approach has been taken to simply validate 
some measurable results or actions or answers by comparing 
them stati stically with later judgments such as ratings, 
promotions, survival, and so on, the cost of administering 
and directly interpreting is relatively low. However, if 
the approach taken toward interpretation of the situational 
or job-sample tests is one of a commonsense review by 
management as to the adequacy and appropriateness of the 
actions or performance, or if it is an interpretation by 
psychologists of personality characteristics, in each of 
these circumstances it may be quite costly because of the 
hourly salary of the key individual making the interpretation, 
that is, cither the manager or the psychologist. 

e. No matter how this technique is to be judged, it should 
be employed only after a number of experimental or pilot 
sessions have been conducted and analyzed. 

Parry [52] offers insights into the reasons 
why simulations seem to be more effective than traditional tests: 

The use of games and simulations enables the learner to 
develop in three major areas. Here are the objectives that 
apply to virtually all games and simulations. 

The participant sharpens his skills of analysis, making 
judgments and experiencing the immediate consequences of 
them. For example, he learns to separate relevant from 
irrelevant variables, establish priorities, identify assump- 
tions, separate fact from opinion, set realistic goals, 
assign value (i.e., weight the facts according to their 
relative importance), distinguish between might do and must 
do, and so on. Let's summarize this objective with the 
phrase Analytical Thinking . 

The participant gains insight into his own behavior as 
he interacts with others; and he develops sensitivity to 
the perceptions, needs, goals, management styles--in short, 
the behavior- -of others . To this end, a game might be 
regarded as a projective device. The participants are 
caught up in the spirit of the game and play it without 
stopping to ask, "I wonder what this move will show about 
my personality? How does my performance in the game relate 
to my performance on the job?" Once the game is over and 
these questions come out in discussion, the participant 
begins to realize that the behavior he projected during the 
game is, more likely than not., an accurate sample of his 
behavior on the job. We might summarize this objective 
with. the phrase Insight and Sensitivity . 
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The participant practices and refines his skills in 
dealing effectively with others. Broadly speaking, we 
can divide the work of managers and supervisors into two 
broad areas: people-hand ling skills (c.g., communications, 

human relations, training and developing others, etc.), and 
task -handling skills (e.g., work simplification, planning, 
scheduling, controlling, etc.). Both of these elements 
arc present in most games. A successful manager, of course, 
is effective on both dimensions. In contrast, the new 
manager tends to be either task-oriented or people-oriented. 
Thus, we might regard the games as a series of exercises in 
which each participant can successively develop and refine 
his mixture of people-handling and task-handling skills. 

We could use the phrase Skil ls Practicum to summarize this 
obj ective . 

These three objectives were brought home very nicely 
by a participant who remarked, "These games are sneaky. I 
play them just like I play poker or any other game ... I 
play to win, without giving much thought to what 'style' 

I'm using or what my alternatives are. But then, when we 
sit down to discuss the games, I begin to see some of the 
other ways that I might have played them. In fact, I now- 
find myself giving a good deal more thought to my poker 
game. I guess, in a w f ay, that all of life is a game, and 
that how you play it is just as important as whether or not 
you win. We can't all be winners ... at least not all 
the time. But we can play a good game all the time." 

Greenwood and McNamara [28] report their study 
on "Interrater reliability in situational tests.” Not only 
did they find acceptably high interrater reliabilities, but, 
more importantly,: "... that adequate reliability can be 

obtained from the use of nonprofessional evaluators in business- 
oriented situational tests." 

Proceeding to articles concerned with group 
exercises, Streufert, et . al . [60] make the transition with 

their description of "A Tactical Game for the Analysis of Complex 
Decision Making in Indviduals and Groups." Their simulation 
was designed to maximize differences in information-processing 
characteristics . They ascribe the differences found primarily 
to variations in a personality and environment. This instrument 
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is briefly described, along with techniques for analysis of 
results, and information on how to purchase the associated 
computer program is provided. 

Ilelme et. al . [114] report on "Dimensions of 
Leadership in a Simulated Combat Situation." This simulation 
probes different dimensions than the proceeding one. Although 
it is primarily a report of research, the description of the 
simulation would be of value to a military assessment program 
designer. Some idea of the flavor of a combat simulation can 
be obtained from this excerpt: 

After four hours' sleep, the officer was awakened at 
2:30 and told to report to MAAG headquarters. The host 
nation had been invaded with nuclear strikes. His task 
for the next eight hours was to direct, by radio, four 
jeep-mounted teams in a reconnaissance of war damage and 
radiation levels, continually receiving and recording 
information, transmitting orders to meet emergencies, and 
making a report of results of the survey to his superior. 

Of all the technical and administrative tasks, this one was 
carried out under the most sustained pressure and fatigue. 

Jaffee's book [90] contains concise but com- 
plete examples of both individual (in-basket), and group 
(leaderless group discussion and business game) simulations. 

The examples are complete with scoring check-lists and forms. 

One of the sources most relevant to military 
applications is the report on "Development of Leadership Assess- 
ment Simulations," by Olmstead et. al . [122]. This is a detailed 

description of the design of the simulations currently used 
in the USAIS Assessment Center pilot project. This work was 
done under contract by the Human Resources Research Organi- 
zation (HumRRO) Division No. 4 which is located at Fort 
Benning. The combination of design philosophy, problems 
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encountered and their solution, rating, analysis of results 
and assessor training, make this reference especially valuable 
as a guide for a program designer. 

The in-basket exercise is the most widely used 
individual simulation and the literature is replete with 
articles on this topic. The stature of the exercise is illus- 
trated by this quotation from Byham and Pentecost [12] : "How- 

ever, one exercise, the in-basket, seems to have emerged as 
the most important exercise of most assessment centers." This 
view represents a consensus expressed by other writers. 

Frederiksen, et. al . [22] provide a description 

of how the exercise came into being and its early development. 
Examples of in-basket materials, scoring and validation studies 
are reported. 

The landmark article in the in-basket field is 
Lopez' "Evaluating executive decision making," [119]. This 
source covers the entire spectrum from design considerations 
to application and analysis. It also covers the previously 
mentioned study along with others done by: The Bell System, 

the Bureau of Business, Harvard Business School, Port Authority 
of New York, Sears Roebuck, and General Electric. In scope and 
detail, it is unmatched. 

Meyer [48] details the design and validation 
of the General Electric in-basket. Although the test itself 
is not included, the article would be an excellent resource 
for the program designer. 
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Jaffee offers two books containing in-basket 
information [90], [89]. The latter is devoted exclusively to 

the in-basket. It is a workbook containing two complete in- 
basket exercises with all forms, instructions, etc. to enable 
administration to individuals. This is not to imply that it 
is intended to provide the program designer with an off-the- 
shelf instrument, for the point is made repeatedly that to 
be effective the in-basket must be tailored to a specific 
program. Rather, the book was intended as a training device 
for students. 

f. Reliability 

The summary of reliability studies done by Howard 
[34] negates the requirement for considering them individually 
She writes: 

In many assessment center exercises and in the final 
evaluations each participant is evaluated by more than one 
assessor. Accordingly, interrater reliability becomes a 
matter of some importance, in addition to the reliability 
of individual measures. A summary of reliability data is 
shown in Table 1. 

It should be noted that the AT§T studies of the in- 
basket, projective tests and interviews probably had 
inflated reliability estimates, since the interrater relia- 
bility was determined for the written report of a procedure 
not the procedure itself. Two raters in high agreement on 
what a report says is a far less potent finding than two 
raters in high agreement on how a candidate performs in a 
situational exercise. The reliability coefficients do 
indicate, however, that the reports presented clear evalu- 
ations from which consistent ratings could be made. 

In summary, based on the data available, interrater 
reliabilities for assessment evaluations and for several 
assessment components seem sufficiently high to support 
their further use. There appear to be no advantages of 
ratings vs. rankings or psychologists vs. managers in terms 
of reliability. 
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g. Validity 



Byham's article [11] provides a basic primer on 
validity in laymen's terms. At the other end of the spectrum, 
Taft [61] provides a wide-ranging, probing treatment of the 
subject. With respect to the criterion problem he writes: 

All assessment programs involve studies of the link 
between two or more pieces of behavior, whether the primary 
purpose be selection, validation, research on tests, or 
personality research. Some of this behavior. is known as 
assessment behavior and some as criterion behavior. These 
concepts are analagous to the independent and dependent 
variables in experimental psychology, and it is an arbi- 
trary decision by the experimenter which one is designated 
which. Most of the reports of assessment have devoted some 
space to the criterion problem . . . 

A special problem that arises in personality assessment 
is the frequent unreliability of the criterion which so 
often represent subjective judgments that vary from one 
criterion rater to another. This unreliability imposes a 
serious limitation on the potential validity of personality 
assessments, and it makes it difficult to evaluate some of 
the low validity coefficients reported . . . 

The assessment strategy should be aimed at the criterion, 
once the latter has been established . . . 

A complication that arises in criteria analysis, . . . 

is that an assessor can only predict to indices of the 
criteria, not to the actual criteria, themselves . . . 

Sometimes the assessors may be able to convince those 
who control the criterion ratings that the indices which 
the latter are using are not consistent with their funda- 
mental criterion, but eventually the assessors and the 
criterion raters must agree on some criterion index in 
accordance with the policy of the organization . . . 

All other things being equal, the best assessors for 
predicting existing criteria are those who are partially 
contaminated with the same experience, standards, and 
outlook as the criterion raters and can thus rely on a 
global strategy to make their predictions. 

Taft also addresses the fact that criteria are dynamic rather 

than static: 
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A further condition that is often ignored in assessment 
is that of effluxion of time; the predictions are usually 
made on the assumption that the status of the candidate 
on .the relevant variables will remain constant over time 



In some complex situations, in which the criterion 
performance is highly dependent on the conditions, the 
inability of the assessors to predict the specific condi- 
tions that will operate for any particular candidate may 
render the assessments completely invalid. 



Ghiselli 



This latter 
and Haire [24] 



point is restated 
who write: 



and amplified by 



When we validate selection tests, we typically relate 
scores to some measure of job performance obtained during 
a brief initial period of employment. This initial period 
may be as short as the first two or three months of employ- 
ment, and it seldom is as long as the first two or three 
years. Presumably the men are hired for a much longer 
period. The rationale of our validation procedures involves 
matching a predictor variable with a criterion. But, does 
the brief criterion itself predict the one we are interested 
in- -performance over a relatively long period? To be sure, 
criterion scores for the initial period indicate the con- 
tribution workers have made during that time, but this is 
not exactly what we want to measure. At least implicitly, 
early criterion data are taken to stand for ultimate per- 
formance or total performance on the job. The period 
selected for a criterion is usually determined by practical 
considerations such as the availability of data. The 
question to which this paper addresses itself is: Do tests 

validated against such a brief initial criterion maintain 
their validity over a longer period of measured performance? 
There are indications that they do not . . . Worbois [72], 



The practice of using performance data obtained during 
an initial period and letting it stand for ultimate or total 
performance completely ignores the dynamic character of the 
criterion and important changes that are taking place in 
the worker's performance. Over a considerable length of 
time on the job, there is not only a change in average per- 
formance, but also significant changes in the order of 
individuals in their performance. The change in average 
performance may be much more than we are used to thinking: 
we are used to laboratory tasks which can be mastered in 
an hour or so. On the job, performance may improve for 
many months or even years . . . 



In one case, the authors have followed the progress of 
a group of investment salesmen for ten years. During this 
extended period there was 650 percent improvement in aver- 
age productivity and still no evidence of leveling off. 
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Indeed a straight line would fit the whole data well. If 
there arc significant changes in order within such a period 
of improvement, early validation will be seriously mis- 
leading. A good many studies have indicated such a change 
in order. Contiguous stages of learning typically correlate 
much higher than periods separated by some time . . . Fac- 

torial analyses of learning data clearly indicate a change 
in weighting of factors for different periods of the 
learning process . . . All of these things suggest that 

early criterion data do not measure all of the important 
variables in performance. 



With the average performance and rank order of individuals 
both changing, one is tempted to turn to the rate of improve- 
ment as a criterion. If ultimate or total performance is 
our goal, rate may be a useful substitute. To wait for 
final performance is impractical; even maximal performance 
may be so far off that it delays the validation of new 
instruments impossibly. There is some evidence that final 
level is predicted by rate of improvement, so, even though 
it indicates little about an individual's level of perfor- 
mance during the initial period, it may be more indicative 
of final values than the early criterion itself. In view 
of the difficulties encountered with a partial criterion 
which does not predict the ultimate criterion, this study 
will examine, in a particular case, the validation of tests 
for the prediction of differential rates of improvement. 



From their study of 56 taxicab drivers, Ghiselli and Haire 
arrived at the following conclusions: 



1. It is suggested that the dynami 
criterion of job performance presents r 
in using initial measures. If the desi 
ultimate or total performance, there is 
whether an initial criterion measure wi 
good predictor. If not, validation aga 
criterion is no validation at all. 



c nature of the 
eal difficulties 
red criterion is 
some question 
11 itself be a 
inst this early 



2. Data are presented showing, in a particular case 
job performance over a period of time. During this time 
significant changes occurred in the average performance, 
the variance, and the rank order of workers. 






3. Validating a battery of tests against the criterion 
data obtained, it is possible to predict the first three 
weeks of performance, or the last three weeks, but it is 
not possible to predict both with the same battery. The 
tests show various patterns of validity: low validity in 

initial periods, rising later; high validity in initial 
periods, falling later; and cyclical variations in valid- 
ity measured against successive stages in performance. 
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4. In these data, it is possible to predict rate of 
improvement with some success, but, over the brief period 
investigated, rate is not strikingly associated with total 
production. 

Bray and Moses' [76] review of the literature 
produced these comments regarding the criterion problem: 

Such heightened concerns about test validation led 
naturally to the "criterion problem," a recurring problem 
through the years because few researchers directly face 
up to this issue. A little progress was noted. For example, 
supervisory ratings, one of the most easily obtained cri- 
teria measures, again were convincingly discredited, which 
no doubt will discourage few from using them in the future. 

An implicit point also under fire was whether criteria used 
in various test validation studies were actually pertinent 
to the test validated. 

Another important issue in the past three years is related 
to criteria for selection in a different way. This is the 
question of how long after selection a criterion measure 
should be taken. This concern, as well as a somewhat 
newer interest in developmental changes in employees has led 
to even more urgent pleas for longitudinal studies than was 
true previously . . . 

One of the more impressive analyses was presented by 
Ronan and Prien . . . [123] . . . who appear to be moving 

us closer to a theory of criteria . . . Their review of 

the literature demonstrates that it is totally unrealistic 
to assume that job performance is unidimensional. A single 
criterion, even a composite one, is often very misleading. 

. . . many investigators have not been attentive to the 

multidimensional nature of job performance. Instead they 
have too often used existing, readily available organiza- 
tional indices alleged to be indicative of performance 
rather than devising measures focused on appropriate 
behavior. . . 

What we are saying is that some very serious questions 
must be raised about the efficacy of the continued and 
uncritical use of supervisory ratings as the criterion 
measure. Perhaps this is why we have a criterion "problem." 

Meyer in his article: "The validity of the in- 

basket as a measure of managerial performance" provides a 
description of test design and validation. lie also performed 
a cross validation in his study. 
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Guion and Gottier [29] provide a review of "Validity 
of personality measures in personnel, selection" for the twelve 
year period 1952 - 1963. They present a tabular summary of 
95 validation studies done on fifteen different personality 
and interest inventories. Similar summaries are included on 
projective measures, special inventories, and personal history 
data. Unfortunately, their conclusions are not very encouraging, 
as they note : 

There is no . . . evidence in the survey that companies 

. . . have . . . recognized the need to validate the instru- 
ments chosen for their own specific situations . . . 

Even without these flaws nearly every study may be 
criticized as following the traditional paradigm of testing 
for a relationship between the test and a criterion. Cer- 
tainly more imaginative designs might well reflect the 
undoubtedly complex relationship between motivational 
variables, situational variables, ability variables, and 
behavior. 

They also echo an often-mentioned bit of advice for the program 
designer: 

Whatever the reason, the point still remains: a home- 

made personality or interest measure, carefully and com- 
pletely developed for a specific situation, is a better 
bet for prediction than is a standard personality measure 
with a standard system of scoring. 

This review of references concerned with validity 
is concluded by Howard [34]. She provides tabular summaries 
of several validity studies, as follows: 

What must be regarded as "The Study" in assessment 
center validity is AT&T's Management Progress Study . . . 

which was predictive and "uncontaminated;" i.c., results 
were retained for research purposes only and not released 
to management to influence promotion decisions. The 
researchers administered the assessment procedure to 422 
male employees of six Bell Telephone companies beginning 
in 1956, stored the results, and waited eight years before 
pursuing information on the assessee's progress in the 
company. While many may view eight years of waiting as 
an almost unbelievable display of forebearance , the authors 
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admit tliat by their own standards they were impatient -- 
they had intended to wait ten years. Comparisons made in 
1965 of management level achieved by men assessed six to 
eight years previously are shown in (Figure 9). Validity 
for the assessment predictions was amply demonstrated. 



Figure 9. 



Relationship Between ATfiT Assessment Staff Prediction 
and Management Level Achieved 



Prediction if make 




Status in July, 


1965 


middle management 


% 1st level 


% 2nd level 


% middle 


within 10 years 


management 


management 


management 


Yes 




College 3 




(N = 62) 
No or ? 


2 


50 


48 


(N = 63) 


11 


78 


11 


Yes 




Non-College 3 




(N = 41) 
No or ? 


7 


61 


32 


(N = 103) 


60 


35 


5 



Note: Adapted from Bray and Grant [10]. Source: Howard [34] 

a 2 y 

x significant at p <^.001 



Point biserial correlations were .44 for the college 
group and .71 for the non-college group. Of the total number 
of men who reached middle management, 78 percent were correctly 
identified by the assessment staff. In contrast, among those 
in both groups who had not progressed further than first 
level management, the assessors predicted that 95 percent 
would not reach middle management within ten years. Note 
that these predictions still had two years to run; later 
communications from the company indicate that even greater 
accuracy was achieved. 



Correlations between assessment 
tiveness and salary increments were 
samples of individuals who had at le 
in management since being assessed; 
men, r = .41; (b) Company C, 27 coll 
Company.B, 83 college men, r = .45; 
college men, r = .52. All correlati 
at p < .01. Combined with the data 
ness of AT&T's overall assessment ra 
management success seems well establ 



ratings of general effec- 
also given for four 
ast six years of tenure 
(a) Company A, 54 college 
ege men , r = .51; (c) 

(d) Company C, 39 non- 
ons were significant 
in Table 2, the useful - 
tings for predicting 
ished . 
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Another ATfjT study with newly hired candidates for sales 
positions also used an uncontaminated, predictive validity 
paradigm (5). The primary criterion of performance was a 
six-month field review by an experienced team from AT$T 
headquarters which regularly makes such inspections. Where 
the assessment judgment was "more than acceptable," 100 
percent of the salesmen met the review standards. Comparable 
success figures for those judged "acceptable, " "iess than 
acceptable," and "unacceptable" were 60 percent, 44 percent, 
and 10 percent, respectively, producing a chi-square value 
of 24.19 (p < .001). Again the predictive validity of 
AT&T's overall assessment was evident, this time for the 
job of salesman and over a short time interval. 



Problems of criterion contamination have confounded 
predictive validity studies other than those described 
above, since assessment ratings were used in promotional 
decisions. Where assessment ratings were used primarily 
to make the first promotion, the effect on later promotions 
was not felt to be large, however. A summary of these 
contaminated criterion studies is shown in Figure 10. 



From the studies done to date, overall ratings of 
potential or performance from assessment center procedures 
generally have shown impressive predictive validity, 
especially for managerial jobs. Unfortunately, use of 
the ratings for decision making about assessees ' careers 
somewhat restrains an overwhelming acceptance of the 
findings. Nevertheless, "clean" predictive validity has 
been demonstrated, but only in two studies with both at 
the same company. 

h. Peer Ratings 

Kraut [39] reports an IBM study on peer ratings 
conducted on samples of managers and executives. In summary 
he writes: 

In the business world, as elsewhere, people regularly 
form impressions of their associates. The use of peer 
ratings among managers may merely formalize a process in 
which managers already engage informally. This study indi- 
cates that the use of managers' peer ratings is likely to 
raise the caliber of men who move into the executive ranks. 

A source which deals with a more relevant appli- 
cation is that by Hollander [33]. His paper reports a study 
conducted on U. S. Naval Officer Candidates circa 1955. He 
reports : 
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Summary of Validity Studies of Overall Assessment Rating 
Where Ratings W r ere Used for Promotions 
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Moses AT§T a) 2 or more 

(N = 5,943) promotions 

b) Management level 



Figure 10. (cont. 
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Source: Howard [34] 



The evidence presented uniformly supports the con- 
clusion that peer nominations used early in training can 
make a distinctive contribution to the prediction of a 
long-range criterion of performance after training. A 
direct implication of this is to encourage the judicious 
use of such early evaluations as a supplement to other 
predictive measures. 

Another military application of the use of peer 
nominations (PNs) is reported by Amir, et . al . [2]. Their 

study was done with soldiers of the Israel Defense Forces in 
1964 - 1965. A total of 3897 subjects were involved. Their 
analysis of the results were summarized in the following 



paragraphs : 



What explanation can be found for the high PN validity 
coefficients reported here in contrast to those reported 
in earlier studies? Moreover, what explains the higher 
coefficients reported for promc 
NCO courses than those predict: 
from basic training? . . . In i 

the following reasons appear rc 
tigators observed a high level 
by the soldiers who responded to 
attitude may be a function of 
if not by all, of the troops 1 
commanders may be called upon at any moment to demonstrate 
their leadership abilities in actual combat. Therefore, 
the men tend to choose those of their companions who really 

possessing these qualities. Incidentially , 
of task orientation may be inferred 
the lack of contamination of the PN scores 
choices. The validity coefficient of PN 
scores after friendship scores are partialled out was .55 
as compared to .58 for basic training 
the friendship score 
courses . 



ion to 


officer rank 


from 


g promo 


tion to NCO 


courses 


spect to the former 


question , 


evant : 


The present 


i nves - 


f task 


orientation- 


displayed 


the PN 


questions . 


Their 


e knowl 


edge shared 


by most, 


t their 


present or 


future 



impress them as 
this high level 
indirectly from 
with friendship 



out 

for 



of 

NCO 



and .78 



pri. 


or 


to the partially 


as c 


onip 


ared to .79 


the 


re 


datively high 


the 


f a> 


ct that soldiers 


rite 


ri a 


for promotion 


ears 


to 


be widely 


suit 


of 


the close and 


oldi 


ers 


of all ranks. 


e communication between 



ranks. However, these reasons appear to be more potent for 
troops in NCO courses than for those in basic training, 
which is perhaps why predictions regarding promotion from 
NCO to officer rank are of greater validity than those from 
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basic training to NCO rank. Troops in training for NCO 
rank appear to identify more closely with the values and 
requirements of the army than do the recruits, and are 
already an integral part of the military leadership. Con- 
sequently, they have more understanding of the qualities 
and demands of leadership than do newly enlisted troops, 
so that the PN choices of the NCO trainees are likely to 
have greater predictive power than those of troops in basic 
training . 

The final selection chosen relates directly to 
the assessment center use of peer ratings. Finkle and Jones 
[ S 5 3 provide a discussion on the use of such ratings from a 
design standpoint and cover some of the advantages and dis- 



advantages of 



including peer ratings in 



an assessment program: 



As in situational exercises, we expect that peer ratings, 
if they are to be used, have to be constructed for any 
assessment program in accordance with the format and objec- 
tives of the program. The peer, or sociometric, ratings 
can provide the assessment committee with information as 
to how the assessees themselves judged or viewed one another. 
Such information can be solicited at the end of the program, 
at times during the program, or both. Questionnaires can 
be designed to cover opinions about particular parts of the 
program, such as a group exercise just completed, or about 
overall skills or characteristics witnessed throughout the 
program. The manner of questioning can be as varied as 
any form of rating including asking assessees to rank one 
another (including or not including oneself) on overall 
contribution to an exercise or on specific character istics 
asking them to write out opinions or explanations about 
their own performance or that of others, asking them to 
rate all other assessees on scales similar (or not) in 
content to the variables the assessment committee will later 
employ, or asking them to nominate one or more persons 
from the total assessment group as being the best in certain 
areas . 



Obviously, the perspective of the assessees will not be 
like that of manager observers in several respects. Most 
importantly, they will simply not have been paying as much 
attention to one another as to themselves. This means they 
really have a limited basis for rating one another. Then, 
too, their judgment as to the significance of certain actions 
and abilities may be less realistic than that of experienced 
managers. Finally, peer ratings are always somewhat suspect 
to popularity contests (despite some clear research evidence 
to the contrary) -- though absence of exposure to one another 
prior to the assessment program should forestall this pos- 
sibility considerably. 
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i. Assessors 



A thorough discussion of the selection and training 
of assessors is provided by Byham [11] who writes: 

Choosing the assessors 

Typically, assessors are line managers working two or 
three levels above the man being assessed. . . . These are 

the individuals who are responsible for promotion and who 
know most thoroughly the job requirements of the positions 
one level above the candidate's. 



The job background of the assessor, of course, depends 
on the purpose of the specific assessment center. Where 
broader management aptitudes are being assessed, it is 
common for the assessors to be drawn from a number of areas 
in a company. This not only brings in a number of viewpoints, 
but exposes the candidate to representatives of a number 
of areas where he may find promotional opportunity. Having 
representatives of different areas also increases the 
acceptance of the findings throughout the company. 



Assessors from management, like 
selves, are usually nominated by th 
in a few companies the center admin 
effort to recruit them) . Naturally 
dangers. After a center has passed 
to the operational phase, "purity" 
somewhat, and senior management may 
"cooperative" managers to centers t 
This temptation is particularly str 
serve for extended terms. 



the candidates them- 
eir superiors (although 
istrator makes an 



the practice 
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Center administrators have chosen to 
problem in various ways. Some companies 
assessor training programs to screen out 
are unacceptable in the role, for one re 
The rationale here is that it is easy to 
fied assessor during training and ease h 
bloodshed. As a fine point of strategy, 
center administrators suggest that it is 
a pool of assessors, rather than train a 
specific assignments. With the pooling 
is easy for the administrator to bypass 
assessors . 



react to this 
rely on their 
assessors who 
ason or another. 

spot an unquali- 
im out without 
for example, many 
wise to establish 
sssess ors for 
arrangement, it 
unqualified 



A major point of controversy among operators of assess- 
ment centers is the desirability of using professional 
psychologists rather than specially trained managers as 
assessors. Most arguments for using psychologists are 
based on their skills in observation; they are trained 
to recognize behavior not obvious to the untrained eye. 
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While this argument is plausible, it has yet to be demon- 
strated in an operational center. Three studies have found 
no differences. 

However, the superiority of psychologists over completely 
untrained managers is well established. Because of this 
superiority, companies often use psychologists as assessors 
in experimental or pilot programs, where training management 
assessors would be difficult. Psychologists are also used 
extensively for assessing higher levels of company manage- 
ment; at high levels, it is difficult to get and train 
managers who do not know the candidates personally, and 
the objective, independent psychologist is seen as the 
fairest evaluator. 

By and large, companies now prefer to establish a pool 
of trained manager-assessors, each of whom serves more than 
once. Individual assessors are usually drawn from the pool 
to serve once or twice a year- -a few companies ask assessors 
to serve only once. AT&T's practice is exceptional -- it 
assigns assessors for six-month terms and center admini- 
strators for one year. 

There are advantages and disadvantages to brief assign- 
ments. On the one hand, brief assignments usually mean 
that better men can be recruited, their enthusiasm and effort 
will be greater, more managers will benefit from the training 
involved in becoming an assessor, and more managers will be 
well prepared, after their tour of duty is over, to make 
judicious use of assessment reports. On the other hand, 
more managers must be trained and kept off their jobs, and 
those who serve briefly will not have as comprehensive an 
experience as assessors as they would if they had served 
a longer period. 

Where the appointment is for an extended period of six 
months or so, of course, more rigorous and lengthy assessor 
training is feasible- -AT^T trains managers for a month-- 
and longer experience in the role is very valuable to an 
assessor. One substantial disadvantage of the long assign- 
ment is that assessment becomes a routine matter, which it 
never should. Reports from fatigued assessors read like 
computer output, and it is hard to think of them as anything 
more. Currently, only ATf*T appoints assessors for prolonged 
periods . 

Training the assessors 

In the companies now operating assessment centers, there 
is a notable difference in the emphasis placed on training 
assessors. Some companies give new assessors as little as 
one hour of training, which really amounts to just an 
orientation to the whole procedure, while most others spend 
three or four days. 
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One can argue that the task of an assessor is similar 
to the requirements of most managers' jobs--a manager must 
interview individuals, observe groups, and evaluate presen- 
tations. Assessing requires skill in these same areas, 
and hence many feel that there is little justification 
for futher training. 



The principal rebuttal to these arguments is this: 
because a man has been doing something, he has not neces- 
sarily been doing it well. Companies reported marked 
improvements in the reliability of supervisory ratings 
after the supervisors have been trained to work as assessors. 
Nonprofessionals need to be shown what to look for in 
observing group discussions and individual, presentations , 
or they may focus on purely surface characteristics. While 
rigid scientific studies are lacking, it is obvious from 
comparing the reports presented by experienced and inexperi- 
enced assessors that training makes a very big difference 
in the quality of performance. 



The most 
In the usual 
an entire ass 
method of ass 
centers are b 
through the a 
Everything is 
present. In 
go through an 
critique the 
observation a 
videotape act 
observations . 



common method of training is by understudy, 
situation, an assessor - in- training sits through 
essment cycle as a nonvoting member. Another 
essor training, particularly when assessment 
eing introduced, is to have the assessors go 
ssessment experience first as candidates. 

the same except that there are no assessors 
a typical training situation, the assessors 
activity such as group discussion and then 
discussion and identify possible areas of 
fforded by the situation. Several companies 
ivities to give assessors practice in making 



foregoing 



Byham and 
information 



Pentecost [12] repeat much of 
and add the following points 



the 



Some companies believe that this experience is so bene- 
ficial that they have gone to a one-to-one assessor-candidate 
ratio in order to expose more people to it. 



Almost all of 
be applied to his 
view and appraise 



an assessor's training 
job and should improve 
his subordinates. 



and experience can 
his ability to inter- 



Greenwood , 



et. al. [28] make 



the following comments 



concerning assessor selection and training: 



. . . the instructions to the evaluators, whether profes- 

sional or nonprofessional , should be explicit as to the 
specific type of behavior to be evaluated, examples provided 
as to kinds of behavior that may be expected to be elicited 
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by the specific situational exercises, and standardized 
rating forms provided for their use. However, the results 
tend to indicate that the selection of evaluators for this 
type of assessment need not be based on whether they have 
completed an extensive training program or have professional 
experience in personnel assessment. 

Taft [61] makes a strong point in support of line 
managers as assessors: 

All other things being equal, the best assessors for 
predicting existing criteria are those who are partially 
contaminated with the same experience, standards, and 
outlook as the criterion raters and can thus rely on a 
global strategy to make their predictions. 

He makes two more pertinent observations in his summary, e.g.: 

The assessors should be selected for proven ability to 
make accurate judgments in the assessment situation, i.e., 
they should be validated. 

The assessors should be familiar with the criterion 
situation, and should take this situation into account 
v T hen they make the predictions. 

Concerning the alleged ’’side benefits" claimed 
for assessor experience, Howard [34] makes the following 
critical observation: 

Training Assessors - -Benefits of assessor training have 
been claimed not only in the form of a partial solution of 
the criterion problem but through (a) improvements in 
interviewing skills (b) broadening of observation skills 
(c) increased appreciation of group dynamics and leader- 
ship styles (d) new insights into behavior (e) strengthening 
of management skills through working with simulations, and 
(f) broadening one's repertoire of responses to problems. 

No wel 1 -desi gned training studies have validated these 
promises, however; as has been pointed out previously, 
firms do considerably more management research on selection 
than on training and development . . . 

In summary, Bender [3] compiled the following 
information from his survey of assessment centers: 
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Figure 11. 

Selection and Training of Assessors 



Number of organizational levels of assessors above assessees : ** 



1 _2 3 4 

Response frequency 0 18 7 4 



Number of assessors 


used when your 


assessment center 


operates : 




3 4 5 


6 7 


8 


9 


11 12 24 






Response frequency 


3 9 3 


12 1 


1 


1 


111- 






Number of days you 


devote to 


train 


ing 


one 


assessor : 








5 hours 


1 2 


2.5 


3 


3.5 4 5 
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9 10 


Response frequency 
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T 3T 


1 
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1 2 6 
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1 3 


**0ne organization 


indicated 


that 


only 


psy 


chologists 


and 


staff 



personnel were used as assessors. 
Source: Bender [3]. 



j . Evaluation of Candidates 

Byham and Pentecost [12] introduce the topic with 
a brief overview of the evaluation process: 

The professionalism of an assessment center is probably 
best measured by forms provided to assessors to record and 
report their observations. These can range from a detailed 
description of all pertinent behavior to a series of l-tc-5 
ratings on key variables. Well -developed forms can guide 
the assessors in making observations and can aid them in 
structuring reports. 

There are several ways of reaching assessors' consensus 
on the candidates, among them a method based on predetermined 
factors. Discussion centers on one candidate at a time. The 
assessor reports to the group on the exercise he has observed; 
all assessors then rate the candidate 1 to 5 on the list of 
factors. This procedure is repeated for each exercise 
reported and the assessors then rate the candidate's overall 
performance on each factor. Next, each assesssor announces 
his rating on each factor and differences are discussed 
until consensus is reached. 
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The advantage of the factor-rating approach is that it 
forces the assessors to consider as important the whole 
range of behavior identified. Otherwise the assessment 
decision might fail to consider impertant areas because 
of overriding positive or negative performance in others. 
Also, the assessment decisions tend to be more organized. 

Another approach is for assessors to give reports on 
a candidate, one after another while all assessors take 
notes. A general discussion follows. This procedure 
facilitates the integration and interrelation of obser- 
vations, but may result in so much data that it is dif- 
ficult for the assessors to consider all of it at one time. 

Still another variation is to have one assessor write 
a composite report on a candidate from individual written 
reports prepared by assessors after each exercise. The 
composite reports are then discussed by the entire group 
of assessors until consensus is reached. 

A more thorough treatment is provided by Tinkle 
and Jones [85] as they describe the procedure used in the 
Sohio program: 

Accepting the philosophy that we learn best by doing, 
we begin the evaluation of our first assessee after having 
spent approximately an hour and a half on orientation. 

With the possible exception of the first case, the assessees 
are considered in alphabetical order. An attempt is made 
by the staff to select as the first case one that will 
probably prove to be rather straightforward and consistent. 

We do not feel this presort is critical and a quick scan 
may not, obviously, accomplish the objective. However, 
the management representatives gain further insight into 
the variables by actually putting them into practice, and 
we feel that it is desirable to avoid the confusion that 
may arise when discussing an unusually controversial 
individual. Of course learning can and does occur when 
there is a good deal of difference of opinion which arises 
because of the complexity of the assessee 's personality. 
However, understanding how to use the variables is dif- 
ficult enough in the first case without adding further 
complications if they can be successfully avoided. 

The same procedural format is used for all evaluations. 

1. Pictures of the assessee are circulated among the 
staff members to refresh memories and to be sure that those 
who have seen the assessees in operation associate the name 
and face correctly. A rating booklet and a completed 
assessment summary sheet containing the relevant test results 
is given to each staff member together with a copy of the 
biographical questionnaire the assessee filled in prior to 
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the program and a set of note paper for making notes on 
reports presented by ether members of the assessment 
committee . 

2. The manager who conducted the personal interview 
of the assessee being reviewed presents his report of 
information which complements the biographical questionnaire. 
He also relates the general manner in which the assessee 
handled himself during the interview. 

3. The psychologist who interpreted the projective test 
materials reads his report. This report focuses upon the 
way the assessee structures and organizes his work, his 
attitudes toward people, his apparent motivation, and his 
emotional stability. 

(Note: Programs employing projective tests are in the 

minority . ) 

4. Then the report of the assessee' s performance in 
each of the situational exercises is presented. Each 
assessee has been observed by a different management repre- 
sentative at each exercise, so that a report on him is made 
by each of the management staff. Responses of the assessee 
to a questionnaire completed after each exercise are read 
to the committee by the program administrator. The tables 
summarizing the peer rankings of effectiveness which are 
completed following each of the exercises are handed out 
with a reminder to keep them through the review of all 
assessees as well as the first. 

5. A discussion is held of the appraisal by all com- 
mittee members of five-minute talks that were given by 
each assessee. 

6. The report forms showing the results of a final 
peer questionanire completed by the assessees as their 
last task during the three-day assessment period are dis- 
tributed- -again with a reminder to keep them through all 
assessee evaluations. 

During the presentation of information, the committee 
members are encouraged to ask questions as each report is 
presented, not only to clarify the specific information 
being presented but also to add further knowledge about 
the procedure and the nature of its contribution to the 
program. This need for a full understanding of the pro- 
cedures is particularly true of the projective test 
report. Most likely, this is the management representatives' 
first contact with the concept of projective testing. It 
is important that they know something of the theory under- 
lying the use and the limitations of this type of testing. 

As the various reports are fully discussed and the presenters 
questioned, the focus of this discussion shouJd be 
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clarification rather than evaluation as such. Either the 
assessee or the particular procedure in use should be better 
known as a result of the questioning. The professional 
staff has the responsibility to see that the committee avoids 
premature efforts to "type" or draw inferences rather than 
give or digest a descriptive report of findings. 



Once the personal history has been reviewed, the inter- 
view, projectives, situational exercises, and questionnaire 
information presented, and the five-minute talk discussed, 
the assessment committee has available to it the evidence 
upon which it will base its recommendations. The task now 
is to draw upon this information to judge the personal 
characteristics in the rating booklet. 



The objective is to build a consensus of opinion con- 
cerning how the assessee behaves as a person as a starting 
point for the committee and other users of the information 
to consider the implications his talents and style have for 
his future contribution to the company. The committee 
members are instructed to rate the scales on the basis of 
their judgment after considering all the information. 

They proceed one scale at a time. The individual members 
make their ratings independently and announce them upon 
request of the committee chairman after all have had full 
opportunity to arrive at their scores. Each committee 
member simply calls out, upon request, the scale point 

( fnr oypmnl o 9 9 ~ 1 / 9 X ^ i.rVi i rVi h n r- r* V /a A -i-r* ^ i c Ai.rp 

^ -l- v v j — > y 4-> JL f «-> y J « < u j. '-.-LA -i_ j. x u j 

rating booklet. The chairman tallies the ratings and 
calls for a discussion of any discrepancies greater than 
one point in order to assure that all committee members 
are weighing all relevant information, excluding irrelevant 
information, and attaching the intended meaning to the 
variable being rated and the bench-marks (scale descriptions) 
on the scale. If the review of sources and scale meaning 
persuades one or more of the committee to change his mark, 
the corrected mark is made in red and the committee chair- 
man is informed. If no one wishes to change his mark, the 
discussion is closed and the next variable is considered. 

This variance in committee opinion is then reflected in 
the narrative report. 



Because the first case is a learning exercise as well as 
a practical handling of results, it is well to avoid the 
temptation to rush. Time is taken to call attention to the 
various sources of information throughout the protocol 
that support the ratings the assessment committee has arrived 
at. Much of the discussion is focused on the scale defini- 
tions so that the concepts will become better crystallized 
for the management representatives. For, as understanding 
increases, the frequency of differences in ratings which 
occur primarily because of differing interpretations of 
the scales' meanings or the behavior indicated by the 
scale points is reduced. 
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We have found that 1-1/2 to 2 hours can be profitably 
spent on the initial case. The time required for subsequent 
cases falls within the range of 1 to 1-1/2 hours. We see 
no evidence that a reduced amount of time spent on a particu- 
lar assessee has reduced the quality of assessment. Our 
observations, admittedly subjective, lead us to believe that 
the feeling of responsibility of the assessment committee 
members causes them to adjust their pace appropriately to 
the complexity and needs of the case they are reviewing and 
in accordance with the confidence they have in their personal 
understanding and mastery of the system. 



One of the procedural problems we have been alert to has 
been the possibility that the interpretation' and integration 
of the cases may vary as a function of the sequence in which 
they are introduced in the integration session. Yet we see 
no evidence that having an assessee discussed early or late 
in the session has a significant bearing on the quality or 
utility of the ratings or recommendations. Repetition of 
the first case as a thirteenth case has indicated that 
being first did not affect the outcome. Admittedly, there 
have been times when one or more members of the professional 
staff has had reservations about the quality of the committee 
judgments or recommendations with regard to a particular 
assessee. However, we do not see such instances as related 
to sequence of consideration, to length of time devoted to 
the case, or to any other procedural factor that might be 
readily adjusted, improved, or controlled. 



Following the completion of ratings on the various scales, 
a Viewgraph transparency is used to project on a screen a 
summary of the tallies made by the program administrator of 
the committee judgments on each scale. Also on this slide 
is shown- -for the first time- -the "pipe organ" diagrams 
showing the results for that assessee on the mental ability 
tests. When these scores are noted, either one member of 
the professional staff or the program administrator reviews 
for the committee the ratings given on each of the scales. 
Reviewing the rating scales, of course, serves to consoli- 
date the group's thinking about the assessee so that all 
members view him in a reasonably comparable way before con- 
sidering the questions of placement, development, and 
potential. We have adopted the practice of withholding 
information about the assessee 's overall mental ability 
as determined directly from tests until this time to avoid 
the possible biasing effect of this information on the 
judgments of other characteristics and skills. This factor 
is then introduced at the time the judgments are reviewed 
and is considered in the general comments, the placement, 
supervisory, development and potential recommendations. 
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Resolving Differences 



The task of working toward a resolution of those dif- 
ferences of opinion that inevitably arise in the rating of 
the behavioral variables warrants discussion before we 
turn to the process for developing the final recommendations 
concerning utilization and development. In actual prac- 
tice, it is the committee chairman who exercises the key 
role in determining whether an attempt will be made to 
resolve differences successfully, at least to the satis- 
faction of the disagreeing committee members, or whether 
issues will be merely politely debated. However, if the 
debate or discussion moves into a somewhat complex con- 
sideration of personality theory, human behavior, or social 
interract ion , the psychologists present will naturally be 
looked to as the experts who should resolve the matter. 

We mentioned earlier the pressure upon the professional 
staff members to speak competently as experts at times and 
to exercise judiciously their responsibilities to aid the 
management representatives in the interpretation of appar- 
ently contradictory test findings. If the professionals 
are not careful, however, they can misperceive their 
appropriately felt pressure to perform in a professionally 
sound manner as a mandate to operate infallibly. Yet we 
have discovered that a professional staff person can be 
quite incorrect in his reasoning and- unduly selective in 
his handling of the information available about the 
assessee. It is of course expected that this will not 
occur in every case. The greatest danger, however, is not 
whether the psychologist will be wrong in some instances- - 
he will be- -but whether he is willing to admit his error. 

If he cannot, then the session will turn into little more 
than a lecture for the professional staff with the manage- 
ment representatives and other committee members serving 
as a reluctant audience. Essentially, then, we are sug- 
gesting that each psychologist must be willing to assume 
an attitude of inquiry and participate in the search for 
the best possible assessment answers rather than attempting 
to promote his own viewpoints as being professionally 
correct. 

The professional staff sets the tone in this regard. 
Consequently, if the professionals operate more as searchers 
than advocates, it is easier for the management representa- 
tives to also assume this approach. Actually, they are 
inclined to do so anyway and need only the support and 
encouragement that the psychologists' approach can provide. 

Though we have indicated that the management repre- 
sentatives look to the professional staff for leadership, 
this is not to suggest that they are or ought to he 
passive followers. Indeed, our experience has been that 
the management representatives have been quite ready to 
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contribute their own ideas and have been quite perceptive 
concerning human behavior. In short, they have shown little 
tendency to be easily cowed or accepting of prima donnaish 
actions on the part of the permanent staff. Any efforts 
toward resolution of differences have clearly been joint 
efforts. Happily, the focus has not been on "who is right" 
but rather on "what is right." 



Thus a difference in rating or recommendation should be 
approached from the viewpoint that an agreement or con- 
sensus may be desirable, but not necessary or perhaps in 
some instances not possible. Clearly, no one should change 
his opinions unless the available evidence leads him to a 
different interpretation. Most frequently, any disagreement 
in ratings occurs as a result of attending to different 
portions of the information, and the difference is usually 
resolved by one or perhaps both of the disagreeing members 
acknowledging that he has not given the weight to a specific 
piece of information that the variable under consideration 
warrants . 



When a difference does become apparent as the committee 
members announce their ratings, each of the differing 
views is given a hearing. The committee member who has 
deviated furthest from the others is called on first by 
the committee chairman. The committee members are instructed 
to view' this opportunity to explain their ratings as neither 
a sales presentation nor a defense attorney's plea. Rather, 
each should present the evidence for his rating and the 
reasoning he used in interpreting the evidence. He may 
question the other committee members regarding whether they 
had considered the same evidence or if they can point up 
fallacies in his logic. Others may request him to further 
amplify his evidence or reasoning, or if it seems appro- 
priate, may assist him to better understand the meaning 
of the variable in question or the interpretation of the 
various procedures. As indicated earlier, any further 
discussion that moves in the direction of persuasion or 
disparagement is discouraged. 



The chairman should also keep conversation relevant 
by sensing when a discussion has reached a point of dimin- 
ishing returns or has begun to stray too far afield in 
defense or rebuttal of a point of view. In this context 
it is important that the professional resist the temptation 
to lecture to the group. We have already mentioned this 
in pointing out the danger of operating as though the task 
were to present his infallible judgments. Even though 
the professional staff member assumes the spirit of inquiry 
we have described, he is in some danger of operating too 
fully in an instructor role, for frequently he sees an 
opportunity to enlighten the group concerning what he 
feels is an especially valuable psychological insight. 
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We do not suggest that lie always refrain from following 
up such an inclination, but we do recommend that he do so 
with exceeding caution lest he sabotage his own efforts 
by- becoming irrelevant or technically overbear ing . 

Before turning to a consideration of the development 
of recommendations, we should mention that most of the 
disagreement in any integration session occurs during the 
rating of the variables rather than in the making of 
recommendations or the evaluation of potential. We feel 
that this is as it should be. The scales have demanded, 
in effect, that the committee recognize different percep- 
tions, discuss these differences openly, and clarify their 
thinking. By the time the developmental, placement, and 
other recommendations are solicited a clear' and more 
uniform view of the assessee generally prevails among the 
committee members. 

Placement Considerations 



In considering the various placements that might be 
appropriate for the assessee, the primary criterion is 
that the placement be one that provides opportunity for 
him to rely heavily upon his areas of strength. It is 
our philosophical conviction that both the individual 
and the corporation are best served by placing him in 
areas in which he is most likely to experience success. 

Common sense, our own experience with the program, and 
anecdotal evidence supplied by successful business leaders 
suggest that the probability of success is greatest when 
the individual is employing strengths rather than trying 
to respond to the "challenge" of positions requiring him 
to surmount personal liabilities. Consequently, any 
other considerations become relevant only if this primary 
criterion is met. 

However, the committee can only provide suggestions 
since a number of factors other than the individual's 
strengths are also considered in placement planning. 

Among these are present corporate needs, the pattern of 
future corporate growth, availability of appropriate 
supervisory climate, and the broadening exposure provided 
by the assignment. 

Although the review of the scale ratings lias pointed 
up the committee's consensual perception of the assessee, 
it is basic that the first step in the consideration of 
placement opportunities be to gain agreement concerning 
which of the personal qualities should be capitalized 
upon. Essentially, this entails only a restatement of 
the assessee's postive qualities in job-related terms. 

For example, assume the individual is organized in his 
work approach, has analytical skill and extensive finan- 
cial training, but has limited forcefulness in interpersonal 
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dealings which results in little impact on others. A posi- 
tion requiring integration of complex financial data but 
with little or no responsibility for supervision or "selling" 
of reports would most likely provide best utilization and 
maximum satisfaction for this individual. Of course, if a 
position could be found that also provides an opportunity 
to gain greater confidence in interpersonal areas through 
being permitted to make well-prepared reports to a group 
which would respect the assessee's "expertise" and bear 
with him in his fumbling but courageous speaking efforts, 
so much the better. However, this would be a secondary 
placement target. 

Once the personal strengths to be utilized have been 
clearly spelled out, the committee turns to the task of 
naming several positions in the company that meet these 
requirements. These positions are listed not as recom- 
mended placements per se, but rather as illustrative of 
the type of opportunity that should be sought for the 
assessee. Such examples help clarify for later users of 
the information the committee's practical interpretation 
of all of the rated qualities. It is desirable to identify 
positions in several different divisions if at all possible. 
Usually, however, the three management representatives on 
the committee at a particular time are not very familiar 
Av'ith a] 1 segments of the company. This further supports 
the concept of considering the committee's placement 
suggestions as illustrative only rather than as compre- 
hensive . 



This is a part of the assessment process in which the 
background of the management representatives is especially 
valuable. They are asked to draw upon their knowledge of 
the different phases of the business to identify positions 
that they feel make the type of demands the assessee's 
native ability, skills, style, and experience have prepared 
him to meet effectively. These positions are carefully 
considered by the committee with the professional staff 
representatives usually taking the role of the devil's 
advocate. It is at this point that disagreement is most 
apt to occur during the recommendation preparation phase. 

It is not unusual for the management representatives to 
mean slightly different responsibilities by use of the 
same term. Areas of disagreement are resolved, or attempts 
are made to resolve them, in much the same manner folloAved 
during the scale evaluation. If the ambiguities of a 
particular position terminology cannot be clarified, however, 
such a position should not contribute to the subsequent 
confusion of others by being- used as an example. 

•Occasionally it is felt by an assessment committee 
that no current corporate positions actually are illustra- 
tive of tow best to employ the skills of a particularly 
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unique assessec. In these instances it might be helpful to 
suggest the design of a new position and the way in which 
it would function. 

Also, it may prove constructive in some cases to provide 
examples of positions that definitely should be avoided 
because the likelihood of failure would be quite great. 

In fact, in a few instances, we have evaluated individuals 
for whom it seemed less important that specific strengths 
be utilized than that some very pronounced weaknesses not 
be employed. Especially for these assessees, it is quite 
worthwhile to take time to delineate the types of positions 
to be avoided . . . 

Developmental Needs 



When introducing the area of development, a good ground- 
work is laid by subscribing to the "self -development" theory 
that no one develops a person but the person himself. Cor- 
porate management is responsible, however, for providing 
the climate and opportunities that will enable the individual, 
if he so chooses, to continue his development in the areas 
into which his experience and initiative have already led 
him. In considering what types of opportunities the corpor- 
ation should provide, it is well to further point out that, 
those qualities that are already strengths should be further 
refined if possible. Weaknesses should be considered only 
to the extent that they are significant impediments to his 
performance. Excessive attention devoted to those liabili- 
ties that do not seriously limit the use of his strengths, 
but rather detract in a more-or-less academic way from his 
being a total person, may not be either a constructive or 
legitimate focus of the corporation. Therefore, in evalu- 
ating the desirability of public speaking training, for 
example, to the extent that such exposure would increase 
an already moderate- to -high level of competence or is a 
necessary skill to function effectively in positions 
realistically seen as options for the assessee, it should 
be encouraged. However, it is highly questionable that 
public speaking training should be recommended and sup- 
ported by the corporation if it might not prove effective 
or simply because it might assist the individual in gaining 
an additional skill when in reality it would be unlikely 
that this additional competence would materially improve 
his likely contribution to the company. 

We believe that the best development the corporation 
can provide comes from job placement that gives opportunity 
to apply strengths. Employing strengths increases the 
likelihood of successful accomplishments which in turn 
improves confidence and often provides greater insights 
into needs for broadening and development. It is upon 
this base that other developmental efforts can build and 
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prove effective. Consequently, some of the most important 
developmental recommendations are contained within the 
section on placement considerations. 

In evaluating further developmental needs beyond those 
satisfied by proper job placement, the committee can be 
given a listing of available formal and informal learning 
opportunities. These might include all of the current 
training programs conducted within the company and also 
examples of external sources and activities which may be 
appropriate. It is not unusual for the committee to 
decide that special developmental opportunities beyond 
those contained within the recommendations on placement 
and supervisory climate are either not required or perhaps 
not warranted in relationship to probable return on the 
dollars invested in providing the individual with a partic- 
ular opportunity. Return on investment is not an easy 
concept to measure when operating in the often intangible 
area of personal development. Nevertheless, it is not 
without merit as a guiding pr inciple . 

A wide variety of specific recommendations may be 
offered. The diversity may run from recommendations of 
a leave of absence with pay to permit completion of a 
doctoral program for a talented researcher to a bibliography 
of suggested readings in management practices for a young 
supervisor. The overriding concern is that the recommenda- 
tion will enable the individual to be more productive for 
the company if he chooses to take advantage of the suggested 
opportunity. Here again, however , the recommendation of 
the committee should be seen as suggestive since many 
factors and conditions beyond their knowledge and control 
bear on the advisability and timing of specific develop- 
mental actions by or on behalf of the assessee. 

Potential 



The final judgment made by the assessment committee of 
any assessee is an evaluation of his potential for contri- 
buting to the company. Again, for this rating a 5-point 
scale with half-step intervals is employed. Opportunities 
(numbers of jobs) at higher levels of management diminish 
quite markedly as one moves upward. Relatively few indi- 
viduals ever move into serious contention for officer 
level positions. And yet nearly all higher level positions 
are filled by individuals who- move from the levels of those 
now being assessed. The committee, therefore, should try 
to estimate the maximum level of contribution as a general 
guide to the company as to its resource potential for 
filling the top levels of responsibility. In actual prac- 
tice, assessment committee judgments of potential, are dis- 
tributed on a normal curve basis. 
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In making the rating on potential, the objective is to 
estimate how far in the organization the assessee should 
advance and still make a constructive contribution before 
the normal retirement age. This may be different from the 
level to which he will actually advance. The rating on 
potential may be higher or lower than the level he will in 
reality attain. Undoubtedly, some men will be advanced 
in the organization beyond the point at which their abili- 
ties enable them ‘to be effective. Others, through circum- 
stances not completely within their control, will not 
advance to responsibility levels they could competently 
handle. It is a major objective of the assessment program 
to provide evidence and judgments that will improve the 
likelihood of each individual's being given the opportunity 
to reach his maximum level of contribution. 

The assessment program will not be without error, 
however. There is surprisingly little disagreement at this 
point in the evaluation, but consensus is not necessarily 
equated with accuracy. IVe simply trust that the careful 
deliberations of a sincere committee will provide general 
guidance that can be constructively used in subsequent 
manpower deliberations . 

After covering current operational methods of 
evaluation, it is considered prudent to review some theory 
on how the process might be improved. Taft [61] provides 
this function, writing as follows: 

Let us conclude this section on the safety in numbers 
assumption with a proposal to combine the advantages of 
both multiple techniques and multiple assessors. The 
suggestion is that each assessor be given a limited amount 
of information on which to base his assessment judgments 
about the candidates, each assessor to receive different 
information. The assessments will then be pooled arith- 
metically. The information supplied may be objective 
or subjective, atomistic or molar, and may range from one 
item to life-history, or a test result, to a projective 
test protocol, an interview or the observation of behavior 
in a miniature situation. This procedure would enable a 
vast amount of data to be integrated without problems of 
weighting since unit weights for each assessor's contri- 
bution would be adequate -- this would be analogous to an 
inventory that gives unit weight to each item. With 
adequate organization of the assessment program, this 
would permit several assessors to contribute to the final 
assessment so that different viewpoints and personality 
theories can be represented. This approach seems to be 
at least worth experimenting with. 
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Even if it is found that increased numbers of assessors 
increases (perceptibly) the accuracy of the assessments, 
there is still a fine calculus of cost in human time and 
effort to be computed. The decision to augment the panel 
with additional assessors is a function, among other things, 
of the gradient of diminishing returns, the ability of 
available extra assessors, the cost of using them, the 
effects on the candidates, and the desire to allow execu- 
tives in the institution to participate in the assessment. 
The proposal made above of having many assessors, who 
contribute small pieces of information, may make it possible 
to conduct multiple assessments comparatively cheaply. 

In summary, Taft makes the following recommendations: 

Each assessor should be given no more than two or three 
units of information; there should be a large number of 
assessors whose predictions are pooled arithmetically, and 
without discussion. 

In selection assessments, if committee decisions are 
desired, the assessors who are particularly well -experienced 
in the criterion situation should be given special influence 
in forming the final decisions, provided they have been 
shown to possess good ability to judge persons. 

k. Feedback of Results to Candidates 

Once again, Byham’s [11] treatment provides a 
brief overview of the topic. He writes: 

One of the most important, yet most hazardous, aspects 
of assessment center operation is feeding the reports back 
to the candidates. Companies handle this in widely dif- 
ferent ways, depending on the purpose of their centers. 

Three companies offer candidates the option of receiving 
or not receiving feedback. Between 60% and 90% ask for 
it. These companies find that candidates who do very 
well and those who do very poorly usually know where they 
stand and do not request feedback, whereas those in the 
middle want to find out how they did and get hints for 
self-improvement. Some companies give feedback to all 
candidates automatically. 

In almost all cases, and certainly in companies that 
are strongly concerned about management development, results 
are carefully couched in terms of the directions that a 
candidate's personal development should take in the future. 
The candidate's impact on his fellow candidates may be 
communicated to him to make him more objective about himself 
His performance on individual tasks may be discussed with 
an eye to establislmig a plan to overcome noted deficiencies 
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When assessment and training are combined, it is pos- 
sible to provide some feedback to candidates prior to their 
leaving the center. In some companies, a candidate must 
wait weeks for a feedback interview. Obviously, the sooner 
the feedback interview takes place, the more impact the 
training and development recommendations will have. 

If a psychologist is available, he usually has the 
responsibility of discussing the center's result with the 
candidate. Otherwise, assessors or former assessors are 
given the responsibility. 

The USAIS [125] provides the following counseling 

guidance to their assessors: 

1. The following is an outline provided to facilitate 
counseling feedback based on assessment exercises. The 
collection, collation and feedback of the information will 
take place as follows: 

A. The Computer Group will provide each counselor a 
packet of information relative to the individual he is 
assigned to (counsel) . This packet will contain a computer 
printout of all rating scales arranged by dimension, and 
the narrative portion from each exercise. A summary sheet 
containing information which relates to each assessee 
derived from paper and pencil tests will also be included 
in the packet. 

B. The information given to the assessee will center 

around the dimensions . . . (specified) . The counselor 

is charged with assimilating the information from the 
various exercises into the dimensions for counseling 
feedback . 

C. The counselor will prepare his counseling inter- 
view in the following steps: 

STEP ONE: Read the narrative portions from each 

exercise to get an overall impression of the assessee. 

STEP TWO: Study the computer printout and paper and 

pencil test results. 

STEP THREE: Prepare a Narrative Report describing the 

individual. This report will follow the dimensions . . . 

(listed). The report will contain, all relevant information 
from the narratives, the computer printout, and the paper 
and pencil test summary sheets. This report will serve 
as the basis for the counseling interview and for ARI two 
year follow-up. 
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STEP FOUR: Prepare the counseling interview. The 

sequence of dimensions covered should meet the following 
criteria: 

a. The first dimension selected should be the strongest 
of the assee's relative abilities. 

b. The initial dimensions should be the easiest to 
present from the perspective of action the assessee can 
take to address his weaknesses. 

c. The dimensions in which the assessee is weakest 
should be in the middle of the sequence. 

d. A dimension in which the assessee did relatively 
well should be reserved to conclude the interview. 

STEP FIVE: Review and select tapes to support the 

interview . 

(Notes: (1) The 14 leadership dimensions assessed are: 

Social Skills (Interpersonal Competence), Communications 
Skills, Adaptability, Motivation, Forcefulness, Mental 
Ability, Decision Making, Administrative Skills, Organi- 
zational Identification, Effectiveness in Organizational 
Leadership Role, Supervisory Skills, Physical Competence, 
Technical and Tactical Competence and Problem Solving 
Ability . 

(2) Step Five refers to the preparation of a 
5-10 minute videotape which the counselor records from the 
videotapes of the leaderless group discussion and the can- 
didate's interview of another role-playing candidate. 
Interviews with the USAIS assessors disclosed that the 
edited videotapes were a powerful counseling tool and were 
often able to dramatically convince candidates who had 
doubted the verbal feedback on certain specific points.) 

Finlcle and Jones [85] again provide a thorough, 
although specific, description. They detail the feedback 
procedures employed in the Sohio assessment program which are 
somewhat atypical in that they use pscyhologists to conduct 
the feedback sessions, where most programs utilize lay 
assessors. In spite of this, the description contains a 
great deal of general information which is applicable to 
any feedback system. They write: 
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. . . two basic problems quickly arose when considering 

the feasibility of a feedback session . . . 

The first of these problems concerns the cost of the 
feedback in both time and money. The other, and even more 
important, relates to the possible misinterpretation of 
the results by the assessee and the consequent damage to 
confidence and desire that may ensue. 

With full realization of the problems involved, we 
decided to follow through with a feedback session for any 
assessees who were interested in requesting one. Weighed 
in our deliberations were the following considerations . 

The cost is not great relative to the investment already 
made in the program. It has been our experience that an 
assessment feedback requires at the maximum an hour and a 
half time and more typically forty-five minutes to an hour. 
When assessees operate in areas remote from the home office 
location of the psychologist, it is often possible to arrange 
a meeting when the psychologist is at a nearby company 
location for other purposes or when the assessee has occasion 
to visit the home office on other business. 

Furthermore, the cost must be weighed against more 
intangible outcomes. It is impossible to calculate the 
amount of resentment toward the company that might develop 
in some assessees if they felt that pertinent information 
concerning their personal qualities was being deliberate])* 
withheld from them by management. The loss of a well- 
qualified individual as a result of such antagonism would 
be a real loss to the company. Also, the cost question 
must be evaluated against the possible gains in personal 
growth that the feedback might foster. We will return to 
this issue a little later. 

The possible negative effects resulting from misin- 
terpretation of test findings or the indelicate handling 
of the reporting of the findings to the assessee can 
adversely affect an assessee and the usefulness of the 
program. Indeed, even though the feedback is conducted 
by a psychologist, the likelihood of misinterpretation or 
improper handling is only lessened- -not eliminated. Yet 
there is no assurance that distortion and misinterpretation 
do not also arise in the absence of a feedback. Obviously, 
each of the assessees speculates, at least privately, about 
his own abilities vis a vis the other participants during 
the various group exercises in which he is involved. How 
realistic these self-appraisals are depends upon a number 
of things, including the maturity and insight of the 
assessee, his previous experience in similar situations, 
and his confidence in his capabilities. 
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Without the opportunity to test his own interpretations 
against those of the assessment staff, the assessee may well 
err either in the direction of being a bit too optimistic or 
pessimistic concerning his future prospects with the company. 
In extreme cases he could develop a totally unrealistic 
outlook regarding his potential for advancement. Thus the. 
feedback should further the ability of the assessee to better 
analyze and interpret his performance and the possible 
implications of this relative to his career aspirations. 

The goals of an effective feedback therefore are twofold: 
(1) to provide the assessee with an opportunity to compare 
self-evaluation of his performance with the interpretation 
of the assessment staff and (2) to provide him with the 
appropriate amount of feedback information in a growth - 
oriented context that makes the information as acceptable 
and valuable to him as possible. 

Initiating Self -Development 



The major portion of the writings of psychologists and 
psychiatrists supports the view that the starting point in 
any self -developmental effort is a willingness on the part 
of the individual to examine himself and determine his 
current abilities as carefully as possible as a foundation 
for formulating personal goals involving change. To be 
sure, the task of personal growth is at best difficult 
to govern and achieve. For in addition to a motivation 
to better fulfill his potential capability, an individual 
needs a means of seeing himself as clearly as possible as 
he is. Actually, the assessment, program offers assistance 
in this respect in at least two major ways. 

First, there is considerable opportunity during the 
course of the three days of assessment to critically examine 
oneself. On the measures of intellectual ability, the 
assessee comes to grips with the fact that he can handle 
some types of intellectual tasks more comfortably than 
other types. Many of the attitude and personality inventory 
tests force him to examine his beliefs and his personal 
style. Also, the personal interview confronts the assessee 
with the question of personal goals and his evaluation of 
his own strengths and liabilities. In the group exercises 
the assessee has the opportunity to see how other young 
businessmen tackle problems and conduct themselves in a 
competitive climate. Finally,' when completing the final 
sociometric questionnaire at the conclusion of the program, 
he becomes very much aware that not only is he evaluating 
others in terms of their type of personal impact, but that 
they also are evaluating him. Naturally, he wonders how 
he impressed the others during the three days of formal 
and informal interaction. 
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For many of the employees this is the first time that 
they have really stopped and taken a look at themselves 
in depth. Prior examinations have typically focused on 
intellectual characteristics only. From various college 
quizzes and aptitude tests, they have gained some insight 
concerning their standing relative to general population 
or student norms. There may have been some vocational or 
educational counseling at the time of the selection of a 
college and career. With few exceptions, however, they have 
had little occasion to pull back and reflect upon themselves 
as individuals in a broad sense. 



The second means by which the assessee receives assis- 
tance in self-development comes in his opportunity to receive 
a feedback concerning the interpretation of Ins performance 
and a chance to discuss himself and his future with a member 
of the assessment staff. Such a discussion, if well handled, 
can become the basis for more sound career planning and 
provide both a clearer focus and direction to personal 
developmental efforts. In short, the feedback session 
should further the objective of personal growth' for those 
who are so motivated. 

Personal Feedback- -Not Performance Appraisal 

Because a performance appraisal interview and an assess- 
ment feedback both deal with an individual’s behavior in a 
work situation, we can further understand the nature of the 
assessment feedback by comparing the two . If the assessee 
has had performance reviews concerning his accomplishments 
on the job at fairly regular intervals, he will expect the 
assessment feedback session to be rather similar. However, 
this is not the case for the following reasons. 

Basically, there is a pronounced difference in the 
objectives of the two sessions. The goal of a performance 
appraisal discussion is to provide the individual with 
knowledge about how his specific job performance has been 
regarded for a stated period of time, usually one year. 

By contrast, the assessment feedback deals heavily with 
the effects that personal character is tics have on the 
assessee ’ s present and potential capacity to contribute 
productively in a business setting. 

It is widely held that performance appraisal review 
sessions have their most constructive effect when the 
discussion centers about the accomplishment or lack of 
progress on agreed-upon work objectives. Discussion can 
then proceed to a consideration of the implications this 
performance has for the attainment of future work objectives 
specific to the individual's current job. However, when 
the appraisal session moves away from the focus on 
objective results into an examination of the underlying 
behavioral characteristics that made the performance 
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possible, the defensiveness of the subordinate increases as 
the ability of the supervisor to evaluate this area effec- 
tively comes into question. Some discussion of behavioral 
characteristics is perhaps unavoidable, but the temptation 
to revamp the subordinate as a person is difficult to resist 
and there is danger that the session will digress noncon- 
structively from the task of providing knowledge concerning 
job progress and future work plans. 

By contrast, the assessment feedback is less effective 
to the extent that it becomes too related to specific job 
performance. Illustrative examples may be gleaned from 
the assessee's current job performance, but the emphasis 
is clearly upon the person as a contributor to the organi- 
zation in general rather than upon his behavior in a given 
assignment. 

Another factor differentiating the feedback from the 
performance appraisal is simply who conducts it. This 
distinction arises from the different objectives and the 
relative appropriateness of the manager to deal with the 
information to be covered. Most employees accept the 
concept that their manager is a legitimate authority to 
evaluate the results of performance in relation to agreed- 
upon job objectives and responsibilities. Far fewer regard 
their managei'S as holding the prerogative to either assess 
or attempt te change basic behavioral characteristics which 
in turn influence the attainment of work results. Therefore, 
when the manager moves into these areas, the subordinate's 
defensiveness increases. 

Essentially, the reverse is true for psychologists. 

Many question the capacity of the pscyhologist as an evalu- 
ator of the extent of accomplishment of specific job goals. 
However, many probably agree that as a result of training 
and experience the psychologist is likely to have sug- 
gestions concerning change in personal qualities that 
might yield dividends in improved personal satisfaction 
and accomplishment. 

Also the psychologist stands in a different power 
relationship to the assessee than the manager. Although 
the psychologist may be recognized as a representative 
of the company, there is little perception of the psycholo- 
gist as holding any immediate real power over the assessee's 
future as is the case with his- superior. The impact of 
power in a relationship is such that there is an increase 
in defensiveness and a decrease in exploratory and creative 
thinking. Thus even though a manager may have a positive 
superior - subordinate relationship with the assessee, the 
inherent power structure tends to have a dampening effect 
upon the amount of personalized information that might be 
exchanged. Basically, then, both the perceived legitimacy 
of the psychologist to offer suggestions in the area of 
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persona] growth and the neutral power relationship increase 
the likelihood that the assessment feedback will become a 
session in which there is a meaningful focus upon personal 
qualities and formulation of growth-enhancing plans. 

In summary, then, the differences between appraisal 
feedback and assessment feedback in objectives and personnel 
involved should be clear to those involved. However, these 
two activities do have a complementary relationship to one 
another. Both activities are directed toward improving 
performance and providing guidelines for further develop- 
ment. Ideally, both will serve as vehicles for the individual 
to evaluate better his current performance and perhaps 
gain further insight as to how he can capitalize most 
effectively upon the stronger aspects of his performance 
and adjust other attributes if his capabilities and back- 
ground suggest that this is a feasible alternative. 

Here again the manager and the psychologist both work 
to develop and to utilize better the individual personnel 
resources of the company. The manager's emphasis in 
working with the individual is on the "w r hat” and "how" 
aspects of his results. At the same time the psychologist 
can help the individual better understand the "why" of 
his behavior and gain a better understanding of "where" 
he is apt to find satisfying application of his talents. 

Starting the Feedback Session 



Let us turn to the mechanics of the feedback and two 
important questions: How should the session begin? How 

much information should be conveyed? These questions can 
be best handled if we keep constantly before us the pre- 
ceding discussion and remember the objectives of the feed- 
back session. In actual practice the conduct of the feedback 
session created several problems for us early in the program. 

A large portion of these problems arose from the fact that 
insufficient attention was paid to the context of the 
session. 

For example, our early feedback sessions were too much 
concerned with a one-way report back to the assessee of 
major assessment findings. The criterion for a successful 
feedback thus became that of full coverage of information, 
though this criterion was not articulated and explicit. 
Consequently, assessees were sometimes given information 
that was perhaps interesting but of questionable relevance 
to their own perception of their career objectives and 
personal needs. Had we been more fully cognizant of the 
full role the feedback session can play, we would undoubtedly 
have been quicker to see the joint problem-solving nature 
of the feedback session. 
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Another difficulty of oui own making was to approach 
the feedback task with the rather naive assumption that 
all assessees aspire to positions in top management. The 
feedback information was presented to the assessee in the 
perspective of the assets necessary to progress to top 
management. As a result we were often placed in a position 
of indicating to someone who had modest aspirations in the 
first place that his prospects for advancement to positions 
of heavy responsibility were quite slim because of the lack 
of relevant personal qualities. With the advantage of 
hindsight, we now recognize that the position we had worked 
ourselves into was not. unlike that of a talent scout for 
the Metropolitan Opera telling an average church choir 
member that he had little prospect of success as the male 
lead at the Met. Even though the choir member had no 
operatic ambitions, he would be a bit chagrined at the 
appraisal of the scout. Likewise, our assessees who had 
neither the aspiration nor ability for top management 
responsibility were nevertheless discouraged and resentful 
on receiving information couched in a top management frame 
of reference. 

In these early feedback sessions, it was particularly 
painful to anticipate and conduct a feedback with someone 
who had performed poorly throughout the assessment session. 
However, when we changed the feedback from a simple 
reporting of results to a problem-solving effort, even 
these sessions became manageable and productive. 

Fortunately, even those who had done poorly in the 
assessment session either had an awareness of this and 
had adjusted their aspiration level accordingly or did 
not have unrealistically high ambitions initially. We 
have found that when an individual is given an opportunity 
to describe what it is that he wants from his business 
career, very few men express aspirations higher than the 
assessment staff felt they were likely to attain. Once 
the assessee outlines his objectives, It becomes possible 
to relate his performance to these objectives. Also, by 
obtaining very active involvement from him early in the 
session, it is easier to maintain the session as a joint 
effort rather than to try to introduce artificially a two- 
way information exchange such as had characterized our 
early efforts. 

Quite often an assessee states that he wants to progress 
as far as his abilities will permit, which is perhaps a 
subtle way of asking for the assessment staff's rating of 
his potential. In discussing potential it is important to 
stress that his potential depends upon his own talents and 
efforts, upon his exposure to jobs and people, in order to 
best use and prove his abilities, and upon environmental 
conditions, including the need for his talents, the 
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availability of similar talent, and a grain or two of good 
luck. We have found it most effective to concentrate on 
talking about the relative strengths and weaknesses he has 
and discussing, in general, how he can continue to grow 
in effectiveness as a person. 

How Much Information 



The question of how much information to provide is also 
a thorny issue which is best resolved in each specific case 
by remaining alert to the counseling context of the feedback 
and the stated goals of the assessee. The assessment findings 
should be evaluated and integrated in such a way as to meet 
the assessee ' s needs rather than simply be presented in a 
" take - i t -or- leave - it" fashion without such integration and 
interpretation. Little is gained by giving him the "straight 
scoop" without an attempt to relate it to his individual 
desires and experience. Specific performance on a given 
test or exercise may be used to illustrate a broader char- 
acteristic, but the overriding judgments of the committee 
are the principal subject for review. 



Overall, it is valuable to keep the discussion focused 
on a descriptive level which is 'more integrative than 
evaluative. For example, "You seem not to take full advan- 
tage of opportunities to tune in to the needs of others in 
your thinking and action." Inference is made without cor. 
demnation. An idea is offered for discussion and as much 
as possible the session is oriented about what the assessee 
is ready and willing to accept. 



As long as the session is directed toward description 
and interpretation in the interest of self-understanding, 
there is less likelihood of distortion and difficulty. Nor 
is there danger in discussing the implications of present 
characteristics for the type of person the assessee hopes 
to become and possible steps to take or questions to ask 
himself as he moves toward his objectives. However, danger 
does exist in trying to discuss areas that are essentially 
beyond the control and management of the assessee himself 
or the psychologist with whom he is working. 

One of these areas is the question of long-range potential 
as evaluated by the assessment staff. We have already 
expressed our feeling that the information can be most often 
misinterpreted because it is abstract, future-oriented, and 
subject to so many variables outside the assessee himself. 

Yet one of the basic questions the assessee wants answered 
when he requests a feedback is how much potential for 
advancement he is regarded as having. His question probably 
stems from a need for the security of reassurance that Ii i s 
fate is not sealed and his value to the company short-lived. 
Actually, this reassurance can be given to most assessees 



without the necessity of delving with great precision into 
the estimate of potential. The selection criteria for the 
program are such that few if any problem cases arc evaluated. 
Therefore it is almost always possible to say to the assessee 
with sincerity that he has potential for growth within the 
organization. This fact, combined with the general realism 
of the expressed objectives by the assessee, places the 
question of potential in proper perspective. Most important, 
no assessee should be left with the feeling that a ceiling 
has been established for him. Me should recognize that any 
estimate of potential is just that- -an estimate. Many cir- 
cumstances, foremost of which is the assessee' s own appli- 
cation and development of his abilities, can affect his 
potential. 

Beyond the general question of potential, there are 
other limitations that it may be desirable to place upon 
reporting the data to the assessee. Thus specific jobs as 
opposed to job families should not be discussed with the 
individual as possible future placements. It should be 
remembered that specific job titles are developed in the 
integration session simply to exemplify better the type 
of position that would capitalize upon the assessee's 
strengths. These are examples only. When discussed with 
the assessee, special care must be taken to make sure the 
individual understands the illustrative use of the position. 
Otherwise, the individual i.s apt to begin, thinking in terms 
of a specific job and be disappointed if it is not soon 
offered even though a future proffered position may actually 
meet his needs and the company's needs more effectively. 

Likewise, to the extent that the areas of supervisory 
climate and training are discussed, it is desirable to 
discuss only those things over which the assessee has 
control. In speaking of his relationships with authority, 
it is well to avoid any discuss j on of his immediate superior 
unless the assessee brings up the topic. Frequently, the 
psychologist conducting the feedback has information con- 
cerning the superior. Discreet use of this knowledge may 
be helpful in aiding the assessee in his relationship with 
the superior, but most often the psychologist will do well 
to remember that the focus of the feedback is the assessee- - 
not his supervisor. Frequently, the assessment staff 
recommends outside advanced management programs as company- 
sponsored training activities for the assessee. It may be 
useful to advise the assessee to keep himself open and 
receptive to training opportunities his management provides, 
but beyond this there is nothing to be gained by discussing 
specific programs where the assessee does not control the 
selection. However, any relevant activities the assessee 
can initiate fully on his own should be openly and fully 
explored . 
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Feedback- -Oral or Written Report 



Our feedback sessions have been completely oral. This 
has provided us with desirable flexibility and lias permitted 
a full "back-and-forth" discussion with each assessee to 
assure better understanding. However, even with oral feed- 
backs there is the danger that misperceptions may arise and 
go completely unnoticed by the psychologists. There is a] so 
the disadvantage (at least in the eyes of the assessee) 
that he has nothing tangible to carry away with him from 
the session. 



Though we have not employed a written report for assessees 
at Sohio, such an approach could of course be considered. 
Presumably it should be descriptive in nature and similar 
to the one provided for management. 

An extension of the arguments favoring a written report 
for the assessee would lead to the conviction that he should 
simply be given a copy of the report prepared for management. 
Another view might be that he should be given all detailed 
information in terms of test scores, interview reports, 
exercise reports, and so on that were used by the integration 
committee in forming its judgments. 



Our objective in es tablishing the most effective feedback 
appioach lias been to maximize understanding on the part of 
the assessee as to what lias been concluded about him, how 
it will be used, and how it is likely to affect his career 
with the company. We doubt that an open presentation (even 
"explaining") of "raw" data best serves this purpose in the 
light of the complexities of the techniques and exposures 
used to obtain the data and in view of the nature of the 
multiple judgments contributing to the conclusions. A 
written report may be justified but not if it cuts off 
discussion that gives the psychologist himself feedback as 
to the progressive understanding of the assessee in the 
feedback process. Oral feedback, when well handled, seems 
to do the best job of developing full and accurate under- 
standing. 



It should be remembered, however, that we are seeking 
to provide the assessee with an understanding of the signif- 
icance of the information for his career as well as of the 
content of the information itself. Perhaps the best indica- 
tor of this understanding is a reading by the psychologist 
of the assessee 's emotional reaction to the feedback session. 
Signs of disappointment may often indicate a misunderstanding 
either of the information or its significance. Por persons 
who have an excessively unrealistically high level of 
aspiration, discouragement is to be expected. For most 
individuals, however, the overriding reaction to the feedback 
is constructively accepted. Our experience suggests that 
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the persons most likely to feel disappointment arc neither 
those with exceptionally great talent and potential nor those 
with relatively modest or narrow abilities but, rather, those 
with skills, abilities, and interests that are about average 
or a little better than average for the population of 
individuals recommended for the program. This is, perhaps, 
somewhat like the reactions of "B"- level students in school. 
They experience neither the relief of the "C" -level student, 
who may have had some doubts about passing, nor the satis- 
faction of the "A" -student who, though lie may have expected 
it, is rewarded with the highest possible mark. 

There are several ways to help this "level" of person 
achieve an appropriate understanding of the assessment 
information and its significance for him. The psychologist 
must remember that the results of the program confirm that 
the assessee is exceptional in ability and potential. Those 
selected by management to attend were seen as more talented 
and as having greater potential than their peers in the 
work setting. The assessees we are now discussing were 
seen by the assessment committee as having as much or more 
talent and potential than the typical, well-recommended 
assessee. Clearly, the psychologist should focus on the 
confirmation of prejudged ability rather than on the point 
that some have been seen as having even more talent and 
potential than the assessee being addressed. Beyond this, 
it is important to reiterate what should have been told 
the assessee prior to the program and what certainly has 
been pointed out more than once at the program, namely, 
that the primary objective is to establish and reflect on 
the individual's strengths so that, through his greater 
awareness of these strengths and the company's greater 
awareness of them, they can be more quickly and more 
appropriately applied. Finally, it should be emphasized 
that an individual's potential depends upon opportunity, 
exposure, performance, and upon the judgment of one or 
more key people who have the authority to affect the 
individual's progress. The assessment program operates 
primarily to improve exposure. The contribution the 
individual makes in the eyes of those he is working for 
and the opportunity those persons have to increase his 
responsibility are still the primary determiners of his 
career advancement. Each person's future depends upon the 
extent to which someone who feels that person can make a 
stronger contribution in the organization Is in a position 
to do something about it. 

Because of the critical importance of properly 
designed and executed feedback sessions, it is considered 
important to close this section with some additional words 
of caution concerning possible damaging effects which can 
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result. Meyer et. al . [49] report the following findings 



from a study done at General Electric: 

The average G. E. employee's self -estimate of perform- 
ance before appraisal placed him at the 77 percentile. 

"Only 2 out of 92 participants in the study estimated 
their performance to be below average. 

Commenting further on these findings, Thompson and Dalton 

[63] write: 

. . . The probability is, therefore, that for 70% to 80% 

of all technical personnel a comparative ranking would 
be a deflating experience. Certainly, all would feel 
criticized, and those who asked for the reasons for their 
unexpectedly low rating would receive more specific 
criticism. Judging from the G. E. findings, this would 
have a disruptive effect on the performance of most of 
them. In addition, it would have a negative effect on 
their self-esteem. 

The message voiced repeatedly in these, and other, 

articles is that feedback of results is one of the more 

difficult and crucial factors in the design of an assessment 

program. Thompson and Dalton [63] express it well: 

Performance appraisal touches on one of the most emotionally 
charged activities in business life- -the assessment of a 
man's contribution and ability. The signals lie receives 
about this assessment have a strong impact on his self- 
esteem and on his subsequent performance. 

Although the comment concerned performance appraisal, this 

writer considers it to be equally applicable to assessment 

center results . 

6 . The Big Picture: Integratin g the Subsystems 

The need for an integrated systems approach to per- 
sonnel management was mentioned in Section I of this paper. 

The literature supporting that need will now be reviewed. 
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The basic problem is stated thus by Bray and Moses 
[76] : "What is missing is a unified system that integrates 

manpower planning, selection, training, and development." 
Amplifying this, they continue: 

There is a heightened awareness that selection proce- 
dures do not operate in a vacuum but are part of a system 
The appropriateness of a particular selection standard 
depends not only on the way the job is structured, but 
upon the type and duration of training, as well as other 
factors. A systems view of selection appears to be 
increasing in prominence. Perhaps we may one day sec a 
cohesive theory of man and woman at work. 

. . . Most of all, there is a strong probability that 

we may move from selection systems per se to placement 
systems. To do so will require serious integration of 
existing systems, an area which is still in an embryonic 
stage . 



Miller [50] states the need in slightly different 



terms : 



The development of behavioral ly- relevant job criteria 
the comparison between relevant job behavior and an 
individual's previous work experiences, and the emphasis 
on planning an individual's work assignments in order to 
acquire a variety of samples of his job behavior are all 
integral elements of a career mobility program and a 
behaviorally-oriented identification program. There 
should, therefore, be an interdependence between them. 

The combination of an identification program and a career 
mobility pattern program should lead to more accurate 
selection decisions and consequently more effective utili 
zation of managerial personnel. 

Thompson and Dalton [63] address performance appraisal, and 

suggest that a single system is not adequate: 

. . . Resist the temptation to devise one grand per- 

formance appraisal system to serve all management needs. 
Managers do have to make decisions about assignments, 
promotions, raises, and layoffs, and they also have to 
discuss performance with each man. There are different 
and sometimes conflicting objectives in each of these 
activities; tying them into a single rationalized system 
may make the system less than useful for any one purpose. 
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Management’s own needs for uniformity and consistency 
should not be allowed to impose a rigid system that makes 
impractical demands on the dynamic human organization. This 
does not mean that each of these activities should be 
planned in isolation. On the contrary, the entire method 
of handling performance, salary increases, promotions, 
and job assignments must be considered together. 

In explaining their reasons for such a philosophy, they write: 

The appraisal system is a very poor counseling tool. 

It's possible to evaluate a man's performance, but it's 
very difficult to write your evaluation down on paper and 
then defend it to the engineer. Therefore, in the counseling 
session, you try to anticipate how he'll respond to his 
rating three months later. If you think he will be dis- 
satisfied and complain about it, then you only write down 
his faults. You don't tell him he's done a good job even 
if lie has. In fact you don't counsel the man at all; you 
just try to build a case for the rating you will later 
be giving him. 

The appraisal system favored by Thompson and Dalton is MBO. 

They compare it with conventional techniques as follows: 

Writers have advocated a management -by-ob j ectives 
appraisal system for many reasons. Douglas McGregor, for 
example, favored it as a way to help a manager stop "playing 
God"-~that is, judging the personal worth of his fellow 
man. Alva F. Kindall and James Gatza were concerned about 
"quackery" - -noting that under conventional performance 
appraisal, the manager was asked to diagnose personality 
traits. For them, concentration on target-focused appraisals 
assured a healthy emphasis on the task. Herbert H. Meyer, 
Emanuel Kay, and John R. P. French were able to show that 
criticism had a negative effect on achievement of goals, 
and that performance improved most when specific goals 
had been established. 

Slevin [59] offers a model depicting the relationship 
and contribution of the assessment center within the larger 
system : 

The assessment center seems promising as a new technique 
for the selection of future managers. However, the most 
effective utilization of this technique will combine not 
only the appraisal function, but also the management develop- 
mental function of the center. The flow chart shown in 
Figure (12) indicates the way in which the assessment center 
can be used in a two pronged attack on organizational 
development . 
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Figure 12. 



Use of the Assessment Center as an Appraisal 
and Development Tool 




Source: Slevin [59] 

The assessment center is conducted in usual fashion 
with an emphasis on both evaluation and the diagnosis of 
developmental needs. These two variables may then be fed 
into an over-all manpower plan. This plan takes into 
account the environmental demands on the firm and the pro- 
jections of future manpower needs in various areas. Based 
on his own competence and what his developmental needs are, 
an individual will be plugged into two plans: a promotion 

program that is specifically constructed for the individual, 
and a management development program. The promotion pro- 
gram specifies a career path that the individual is likely 
to follow in the organization. The management development 
program focuses on the difference between the individual's 
current competence and future needs as defined by his 
tentative career path. Using both the promotion and 
management development programs, the organization can 
implement an effective and farsighted manpower plan. 

In conclusion, evidence that these concepts are not 
merely theories but can be attained in practice is offered 
by Campbell, et . a l . [78]: 

Practices in this company represent one of the few 
examples we found of a careful blending of validated 
selection methods and the use of informed and systematic 
appraisal methods for administering incentives and tailor- 
making development programs. Thus, the testing, compensa- 
tion, and salary systems as well as the effective use of 
both company and noncompany training programs arc geared 
toward identifying, developing, and motivating effective 
people to become effective managers. 
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No other company we visited had a program the same 
depth of coverage even in single areas that this company's 
practices show in all the key areas. 

7 . Unresolved Problems 

The report of a seminar conducted by the Foundation 
for Research on Human Behavior [112] contains a listing of 
questions for further research. Although this list was 
compiled in 1957, this review of the literature has not 
disclosed significant progress on them and they appear to 
be as timely as they were in 1957. The listing is as follows: 

To what extent is improvement possible in the various 
managerial characteristics that are related to job-effec- 
tiveness? To what degree are managers born and not made? 

Are certain key attributes resistant to change? If so, 
perhaps more emphasis should be placed on selection and 
placement procedures. Are most key attributes capable 
of being developed? If so, perhaps more emphasis should 
be placed on training. 

The future demand for managerial talent will not permit 
organizations the luxury of accepting only men with "no 
weaknesses." In what ways can an executive be helped to 
overcome his weaknesses? What strengths can be developed 
to compensate for these weaknesses? What characteristics 
in supervisors, peers, and subordinates can be sought to 
offset these weaknesses? What modifications can be made 
in a man’s job-description or in organizational structure 
to take these weaknesses into account and yet maximize 
returns from his strengths? 

What procedures can be used in dealing with the "non- 
promotable" man, the man who is adequate in his present 
position but has reached his limit? In what ways can he 
be helped to lower his aspirations without destroying his 
motivation? 

What procedures can be used in dealing with the sub- 
ordinates of the "non -promotable" man? What opportunities 
can be provided for the subordinates to continue their 
development? Should they be rotated through other units? 
Should the organization assign the non-promotable super- 
visor activities which will take him away from his position 
frequently and permit subordinates an opportunity to under- 
study him? How may the tendency for low-grade supervisors 
to attract low-grade subordinates be dealt with? Are there 
ways in which the non-promotable man may be made more secure 
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in his position so that he will be less threatened by 
subordinates and lienee be better able to develop them? 

What changes occur in an executive after he assumes a 
new managerial position? What effects do experience on the 
job and being "in the role" have on his competence? Can 
these post-promotion influences be predicted? 

In studying successful performance, how much error is 
introduced by contrasting "more successful" and "less 
successful" groups of managers? Presumably, some random 
situational factors (non-promotable boss) were responsible 
for the differences in achievement of the two groups. More 
important., however, possibly some systematic factors 
(power tactics?) unrelated to the goals of the organization 
were responsible for some managers being promoted and 
others not. How does one take into account the effects 
of differential mobility in different departments? Does 
test research based on such "success" criteria tend to 
perpetuate errors? 

Are established principles of management development 
fully utilized? For example, are fledgling managers per- 
mitted a sufficient proportion of "success experiences" 
early in their career to enable them to acquire self- 
confidence and maintain a high level of aspiration? Does 
management pay enough heed to the fundamental law of 
learning: heavy on reward, light on punishment? What 

proportion of its management talent does an organization 
waste by using the sink-or-swim method of training managers? 

To what degree does success breed success, and failure 
breed failure? Does success provide self-confidence which 
enables a man to be more imaginative, independent, original, 
and assertive? Docs failure lead to increased fear of 
failure which prevents a man from making decisions, or 
admitting ignorance by asking questions? To what degree 
is success or failure a circular, self-perpetuating process? 
If so, what modifications in management development, job 
assignment, etc., can break the failure circle and rein- 
force the success circle? 

To what extent do managerial jobs impose rigid patterns 
of behavior even when other equally satisfactory ways of 
achieving organi zational objectives exist? One participant 
phrased the question as follows: Is managing like fighting 

bulls or playing baseball? The bullfighter cannot achieve 
success by merely killing the bull. He must pattern the 
smallest details of his performance to the rigidly held 
expectations of a number of people. The baseball player, 
on the 'other hand, can achieve success by maintaining a 
high batting average or by being an excellent ball hawk. 

It doesn't matter whether lie bats with his "foot in the 
bucket," or whether his fielding is awkward. All that 



135 



counts are results. Is an executive judged successful on 
the basis of results alone or must he achieve results in 
a certain way? 

In summarizing her recent study of current assessment 
programs, Howard [34] identified future research needs as 
follows : 

A number of issues have been raised in this review that 
need to be researched, and undoubtedly there are many more. 

A brainstorming session at the Center for Creative Leader- 
ship of the Smith Richardson Foundation produced a list of 
over 100 relatively unresearched questions concerning assess 
ment centers [38] . All the components of the process and 
their integration can be studied more intensively, as can 
the potential uses of assessment procedures with such 
organizational problems as personnel selection, training, 
motivation, job analysis, job satisfaction, and organi- 
zational climate. 

Aside from the as yet unexplored research questions, 
the importance of situation related validation rescarcli 
for those attempting to use assessment centers should be 
emphasized. Because their initial industrial development 
was grounded in research, it is often casually implied 
that therefore any assessment center will work. It is 
true that AT$T laid a solid basic foundation in its 
pioneering predictive validity research. Some other large 
companies have also contributed to the . . . (current 

knowledge of assessment center validity). 

Howard goes on to add that future program designers cannot 

expect to crowd under AT&T's validity "umbrella," but rather 

must tailor both program and validation design to their 

specific situation. She further emphasizes the importance 

of utilizing the feedback from validation to refine the 

assessment program and keep it aligned with the demands and 

dictates of the environment. 

8 . A Critique of the Literature 

I.n the opinion of this writer, the greatest weakness 
of the literature on assessment centers is that it rests on 
too narrow a data base, e.g., although there are a great 
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number of articles, the bulk of them are by a relatively few 
authors and about a small number of programs. It is speculated 
that ATIiT, Sohio, IBM, AMA, Sears, J. C. Penney, and General 
Electric's programs together would account for an overwhelming 
majority of the literature- -indeed, the first three would 
probably contribute 40 to 501 of the total. 

Although the reason for this is the fact that the 
concept is still relatively new, the increase in numbers of 
programs will not, by itself, solve the problem, for quality 
is needed as well as quantity. This is to say that more 
research of the caliber of the Management Progress Study is 
required if we are to resolve the important problems still 
facing us. 

C. DESIGN OF A PROTOTYPE ASSESSMENT CENTER FOR THE NAVAL 
POSTGRADUATE SCHOOL 

1 . Assumptions 

a. Candidates would be U. S. Naval officer students 
in their first quarter of the 817 Management curriculum ef 
the Naval Postgraduate School. 

b. Assessors would consist of a mixture of U. S. 

Naval officer students and faculty of the Naval Postgraduate 
School. The student assessors would be selected from students 
in their fifth quarter. 

c. That sufficient flexibility could be obtained 
in the curriculum to allow tailoring of academic programs 
to individuals in accordance with assessment findings. 
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d. That the primary objective of the assessment 
program would be research (regarding selection, placement 
and development) with a secondary objective of development. 

e. That the program would focus on the managerial 
facet of the Operational, Technical, Managerial Systems (OTMS) . 

f. That research data collected would be privileged 
and not divulged outside of the prototype assessment center 
staff. 

2 . Phil osophy 

a. In perspective, assessment centers would only be 
subsystems within the larger system of personnel management 
within the U. S. Navy. The assessment centers would, of 
necessity, interface with many other subsystems such as 
initial selection, development, placement, selection for 
promotion, and performance appraisal. Of these (hopefully) 
complementary subsystems, performance appraisal -- 1 ike assess- 
ment involves the dual roles of selection and development. 

This writer proposes that appraisal for selection 
purposes should be completely separated from appraisal used 
for feedback and/or development. Research has shown that 
selection ratings are enhanced by confidentiality. Peer and 
subordinate ratings also have potential in this area and are 
worthy of investigation. Several articles deal with this 
subject, including three cited earlier in Section III B h , 

Peer Ratings (e.g., Kraut [39], Amir et . a 1 . [2] and Hollander 

[33]). All of these supported the use of peer ratings for 
prediction of later performance. Githens and bister [313] 
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found subordinate ratings to have potential, but that their 
use was considered threatening to the majority of Naval officers 
surveyed. Ideally, selection appraisals should be specifically 
designed to complement the assessment center results and 
should concentrate on those dimensions which can be best 
observed on the job. 

This writer offers management by objectives (MBO) 
as the best vehicle for the developmental aspect of performance 
appraisal. Articles on MBO abound in the literature, however 
McGregor’s classic [47] remains one of the best. He condemned 
"conventional" performance appraisal and recommended analysis 
vs. appraisal where the superior would act as a coach instead 
of a judge. 

b. In the "ideal" system, as envisioned by this 
writer, more than one assessment would be performed during 
a career and. the objectives of each would differ. The concept 
could be modeled as shown in Figure 13. The assessments 
indicated in the model are explained as follows: 

(1) Career Entry . Since no performance appraisal 
information is available (or at least none in the job being 
applied for) the hiring decision would be based solely on 
assessment data. This process would involve large numbers 
of candidates, thus, would have to be as short as possible 
while still obtaining valid results. The AT(jT one-day Early 
Identification Program described by Moses [51] is an example 
of this type of program. Selection would be the primary 
objective, and the brevity of the program would virtually 
rule out any developmental aspects. 
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Figure 13. 



Number and Nature of Assessments During Career 




(2) Career Commitment . This point would occur 
after four years of service, when the officer had completed 
his obligated service and elected to remain in the service 
as a career. Such a candidate would be in the rank of 
Lieutenant and, if a line officer, would have completed his 
first operational (sea duty) tour and be commencing his first 
tour of shore duty. He would soon be faced with the decision 
as to which career path to choose (Operational, Technical, 
or Managerial), and an assessment should provide information 
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not obtained in his performance appraisals which would enhance 
the quality of this decision- - to the benefit of the individual 
and the U. S. Navy. 

(3) Selection for Initial Command . By this stage 
of a career the difference in career paths make any "norm" 
selected a little less representative, however it should occur 
at about 12 - 15 years of service at a rank of senior Lieu- 
tenant Commander or junior Commander. This decision is a 
crucial one for the individual as well as the organization. 
Since it represents a marked change in the degree of responsi- 
bility and scope of decisions required of the officer, an 
assessment can help combat the "Peter Principle" by probing 
those key factors of command which have not yet been evaluated 
on the job. 

(4) Selection for Major Command/Flag Rank or 
Early Retirement . Once again the degree of 

responsibility and scope of decisions take a quantum jump, 
therefore the benefits of assessment cited in the previous 
section would apply here as well. For officers retiring 
early, an assessment could provide them with an inventory 
of their attributes to assist them in deciding on a second 
career . 

(5) Retirement . Same considerations as early 
retirement previously described. This could be viewed as a 
sort of "Project Transition," and although the direct benefits 
would accrue to the individual, the organization might benefit 
indirectly from such a "personalized" personnel management 
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policy, e.g., such a benefit might make a Naval career much 
more attractive to young officers thereby enhancing the overall 
quality of the officer corps. 

It should be noted that the emphasis is on selec- 
tion initially, but there is a gradual shift toward development 
as the career progresses, until at the end, inventory and/or 
development are the primary objectives. 

Another potential objective is that of placement. 

By ascertaining an officer's strengths and weaknesses with an 
acceptable degree of accuracy, it would be possible to place 
him in billets better suited to his talents--as advocated by 
Fiedler [19] - -thereby making better use of the human resources 
of the Navy. 

c. Assessment programs can be classified according 

to their use of signs and/or samples. "Psychometric" programs 
emphasize signs, while at the other end of the continuum, 
"behavioral" programs emphasize samples. ^ Most current programs 
fall nearer the middle of the spectrum and assess both signs 
and samples. 

d. Design of the assessment program should be based 
on the best available information. In this writer's opinion, 
the best base would be data obtained from a quality research 
effort. An alternative would be to obtain the data from a 
panel of "experts," however the validity of such data would 
be questionable. 



^Ihe author realizes that all signs are samples, but 
whether or not a sample is a sign is a matter determined via 
validation efforts. 
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A decision would have to be made as to whether 
an officer could be selected on specific dimensions unique 
to a billet (or family of billets) or whether the heterogeneity 
and number of billets would force an assessment of the "whole 
personality" on general dimensions as was done by the O.S.S. 
[99]. This decision would properly require thorough job 
analyses which would define the billets and enable judgments 
as to which (if any) might be combined into "job families." 
These job analyses should be behaviorally-oriented to identify 
significant job dimensions which can be measured by observable 
behaviors (for subsequent design of the assessment program) . 

3 . Program Considerations 
a. Objectives 

(1) Primary . The primary objective of the pro- 
totype assessment center would be research focusing on the 
selection and placement of managers. Virtually all of the 
data obtained on this aspect would be held in strict confi- 
dence by the assessment center staff. 

(2) Secondary . A secondary objective would be 
the individual development of participants, both candidates, 
and student assessors. The goal would be to produce a per- 
sonal development plan for each candidate which could be 
translated into an individual curriculum. Although the sub- 
stantive differences between individual curricula might be 
small, the participative involvement of the student in the 
determination of his academic program could produce signifi- 
cant improvement in satisfaction, motivation and, utlimately, 
performance . 
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b. Content 

Por a research-oriented program such as this, a 
mixture of signs and samples is considered to be the prudent 
choice. This would offer a full range of instruments for 
validation and enable the validation process to indicate which 
are most effective. Any restriction of this (i.e., using 
only signs or samples to the exclusion of the other), would 
amount to "pre-selection" decisions without scientific basis. 

c. Length 

In view of the academic schedule which exists at 
the Naval Postgraduate School, this writer considers the 
five-day program to be the best length. This would roughly 
break down as three days of assessment followed by two days 

i 

of evaluation and report writing by the assessors. 

d. Administration 

The academic schedule poses some challenging 
scheduling problems. One solution would be to make the 
fifth quarter of the management curriculum a "vertical" 
schedule for those students selected as assessors (e.g., the 
courses would be taken consecutively vice simultaneously). 
Assessor training could be conducted during the first week 
of the quarter, and then assessment of twelve candidates per 
week conducted until all candidates had been assessed. Since 
the candidates would only be involved for three days, their 
other course work could be scheduled lightly until assessments 
were completed and then the tempo increased to compensate. 
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D. DESIGN OF VAL I. DAT I ON /EVALUATION/ FEEDBACK SYSTEM 



These systems would differ according to the objectives 
of the program to be evaluated. Their design should be 
accomplished at the outset before the assessment program 
design is undertaken as recommended in the O.S.S. "Assessment 
of Men" [99] as follows: 

. . . appraisal is the target of assessment, and it is 

■ always -well to sec your target before taking aim. One 
should know ahead of time, for example, whether the target 
will be an objectively determined figure, such as the 
number of units of work a man completes per day, or a 
rating of his efficiency given by competent observers . . . 

The point that is being stressed here is that whatever 
the determinants of appraisal may be, it is important to 
distinguish them at the start, so that the assessment 
program can be designed to take account of them. 

It must be pointed out that the foregoing tacitly assumes 

the elements of appraisal selected to be relevant to the 

objective(s) of the assessment. 

Once developed, the appraisal system should be tested 
on a number of "successful" members of the organization, 
if possible, and then revised as necessary. 

A final consideration is that the rating scales for the 
assessment and appraisal programs should be identical on 
those dimensions which are shared by both. 



IV. JUSTIFICATION 



A. OF ASSESSMENT CENTERS 

The Management Progress Study, and other AT^T studies 
which controlled for contamination, offer the most rigorous 
proof of the concept. In addition to these, one must acknow- 
ledge the amount of resources devoted to assessment centers 
by large prof it -making organizations. This is not to advocate 
a blind policy of fol low- the - leader , but the size of the com- 
mitments made (5 million dollar annual budget by ATmT [53]) 
are worthy of serious consideration, since no organization 
would commit such resources to a program unless they felt 
they were getting a reasonable return. AT§T has supported 
the concept for seventeen years and continued to increase its 
commitment with each passing year. The British and German 
military programs also represent heavy commitments of long 
standing . 

B. FOR ESTABLISHMENT OF A PROTOTYPE ASSESSMENT CENTER IN 

THE U. S. NAVY 

If one accepts the concept as valid, then the remaining 
question becomes: How to best utilize it within the Navy- - 

vice whether or not it should be. 

Some interesting and prophetic remarks in this regard 
were made by Rear Admiral I. E. Hobbs to students of the 
U. S. Navy Management School at Monterey [315] to wit: 
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Perhaps we should take a good look at what the Army is 
doing experimentally in evaluation procedures to try and 
measure an officer's interest and abilities in three fields 
of combatant, administrative, and technical. This is 
being done by two full days of achievement testing and 
then validated by actual field performance tests two to 
five years later. 

This method of testing is somewhat similar to the 
German method used after World War I and used by the Bri- 
tish and O.S.S. during World War II. It has considerable 
possibilities and should be carefully monitored by the 
Navy as a means of identifying truly outstanding officers 
in the fields of operations, planning and technical branches 



The Air Force, through a special contract with the 
University of California, is conducting some very signif- 
icant tests in evaluation of officers at their Squadron 
Officer School (AF opposite to the Navy General Line 
School) located at Maxwell Air Force Base. This effort 
consists of student peer ratings, situational testing 
during escape and evasion, instructor evaluation and 
other psychological procedures and techniques which are 
available but not generally employed. Through monitoring 
and observing these efforts the Navy might well learn 
additional techniques of identifying naval officers to 
man our fleet in the post 1970 era. 

The prophesy was the mention of the fields of "operations, 

planning and technical branches" which recently became reality 

as the Operational, Technical, Managerial System (OTMS) . The 

irony is that these remarks -- timely as they sound today- -were 

made on 20 February, 1959. In the ensuing years, assessment 

centers have not been disproved or lost favor, but rather, as 

Bray [ 68 ] puts it: "... have reached the critical -mass 

state and are ready to take off as more companies use them." 

This writer submits that the advent of OTMS and the All- 
Volunteer Force have made personnel decisions even more 
important than they were in the past and increase the potential 
value of the assessment center to the U. S. Navy. 
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C. FOR ESTABLISHMENT OF A NAVY PROTOTYPE ASSESSMENT CENTER 

AT THE NAVAL POSTGRADUATE SCHOOL (NPS) 

The Naval Postgraduate School offers a unique combination 
of candidates and assessors from a wide variety of specialties, 
backgrounds and ranks; and necessary supporting facilities. 
These facilities include videotape capabilities as well as a 
full range of other educational media devices, and a computer 
center. Some other significant factors are the expertise of 
the faculty (from Naval officers with operational experience 
to psychologists, psychome tricians , etc.) and the liaison 
with NPS curriculum sponsors for continuous feedback on changes 
in billet requirements. In support of the objective of develop 
ment, the school offers courses in a wide variety from basic 
skills such as reading labs, speech courses, writing, etc.; 
to more advanced work dealing with aspects of leadership, 
interpersonal skills, organisational development, problem 
solving, decision making and human resource management. 

In considering the impact of a prototype assessment center 
upon the NPS, the following three objectives of the school 
are considered to be pertinent: 

Foster an organization and provide supporting incentives 
at NPS in order to optimize utilization of resources to 
meet the learning requirements of the students. 

Provide for continued studies in new programs, new 
techniques of teaching, better definitions of learning 
objectives and measures of effectiveness of our educational 
programs . 

Provide for increased regard, for individual diversity 
and needs on the part of students and provide an improved 
working environment for them. 



This writer submits that significant aspects of all three 
of these objectives could be met by the proposed prototype 
assessment center. These benefits could go far toward off- 
setting the costs of the program. Also, because many of the 
costs represent sunk costs (computer center, educational 
media facilities, faculty, etc.) the actual out-of-pocket 
costs would be relatively low as compared to an industrial 
organization which had none of these. 

In summary, the prototype assessment center could be 
established most efficiently at the Naval Postgraduate School 
and could benefit the school and the U. S. Navy by its 
existence . 
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V. TASKS TO BE PERFORMED 



A review of section III yields the following tasks yet 
to be performed: (The Interviews and Literature Review are 

considered complete as of this writing.) 

1. Selection of criteria to be utilized in the evaluations 
This would have to be done separately for each of the assess- 
ment objectives. 

2. Job analyses of jobs (billets) for which candidates 
would be assessed. Observable behat r iors should be considered 
separately, as they would be of special interest in the design 
of the simulations. 

3. Based on the foregoing steps, a decision must be made 
as to the dimensions to be assessed, and further, on the number 
of different assessment programs which would be required. If 
there is a great deal of heterogeneity among the jobs, a large 
number of separate assessment programs would be called for. 

A more practical alternative to this would be a singK program 
based on "general" dimensions which would attempt to sess 
the Gestalt or whole personality as was done by the O.S.S. [99] 

4. Selection/design of instruments to measure the dimen- 
sions selected (in step 3). In the prototype program consider- 
able overlap would be expected since, whenever possible, com- 
peting instruments would be included to allow validation on 
both of them (e.g., provide selection on a scientific basis 
vice hunch or conjecture). 
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5. Development of rating scales for use in both the assess 
inent and evaluation programs. All dimensions common to both 
programs should have identical scales. 

6. Perform "trial runs" to test all elements of the assess 
ment program which require testing (e.g., instruments designed 
for the program) . These runs should ideally employ "successful 
members of the organization (U. S. Navy) as test candidates. 

7. Make any revisions to the program as dictated by 
results of step 6. 

8. Develop an assessor training program. 

9. Test the assessor training program. 

10. Revise the assessor training program as necessary. 
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VI. ANTICIPATED RESULTS 



A. SHORT TERM (Less than 2 years) 

1. An evaluation of the development objective of the 
prototype assessment center. 

2. Improve the effectiveness and efficiency of the NP5 
by enhancing the attainment of stated goals (as discussed in 
Section IV) . 

3. Increased personal development of student assessors 
and candidates who participate in the program. 

4. Development of an in-house expertise in the design 
and operation of an assessment center. 

B. LONG TERM (More than 2 years) 

1. Evaluation of the selection and placement objectives 
of the prototype assessment center. 

2. If the assessment center proved to be effective, the 
ultimate result could be improved utilization of human resources 
within the Navy. If one accepts the premise that people are 
the decisive factor in combat, the significance of this result 
is obvious . 
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